Config

You can define your problem in JavaScript and paste the url or code in the following area, then click the Minimize button, the minimzed point of defined function will be found.

Log

Optimization log and result will be showed here.

Simple API of JavaScript

The JavaScript will be evaluated with jQuery available. Please define optimization.func, optimization.grad, etc in that JavaScript code.

You can follow an example in this gist, and paste the RAW url in the URLs input area, or, paste the JavaScript code directly in the code area.

Start Point

The start point should be specified through optimization.start. For example,

optimization.start = [2, 2];

Target Function

The target function, whose minimization we want to find can be specified by optimization.func. For example,

optimization.func = function(p) {
  return .5 * Math.pow(1 - p[0], 2) + Math.pow(p[1] - p[0]*p[0], 2);
};

Gradient

The gradient of the target function should be specified through optimization.grad, like:

optimization.grad = function (p) {
  return [(p[0] - 1) + 2 * (p[0]*p[0] - p[1]) * 2 * p[0], 2 * (p[1] - p[0] * p[0])];
};

Solving Algorithm

The solving algorithm should be specfied through optimization.solver. Currently, only three gradient solvers have been adapted, they are LmBFGSSolver, ConjugateGradientSolver and GradientDescentSolver.

TODO: Add some global searching optimization algorithms.

Logistic Regression

As a full example of logisitc regression, we can paste the following two URLs(from this gist) to the above, and click minimize to get the result.

https://gist.githubusercontent.com/zuoyan/81b1b5ab764efe5d9112/raw/0a4c3c53756d6a19c1d444209f8244bb67021bfd/logistic_regression.js
https://gist.githubusercontent.com/zuoyan/81b1b5ab764efe5d9112/raw/7c0495bd14a6c243fe026bee9e543f8b02b97f9e/lr_lowbwt.js

In logistic regression, we assume every feature has some contribution to the clustering, and the final posibility of positive clustering is dertermined by their weight sum:

$$P(X, Y=1) = Logistic\left(\prod_{f\in{features}} w_f \times x_f\right).$$ $$P(X, Y=-1) = 1 - Logistic\left(\prod_{f\in{features}} w_f \times x_f\right).$$

Where Logisitc is a map from real value to posibility range $[0, 1]$, it is defined as:

$$Logistic(x) = \dfrac{1}{1 + e^{-x}}.$$ From the definition, it is easy to verify that: $$Logistic(x) + Logistic(-x) = 1,$$ $$P(X, Y) = Logistic(\prod_{f\in{features}} w_f \times x_f \times Y).$$ The best $W$ is chosen to maximize samples probability: $$P(samples) = \prod_{(X, Y) \in samples} P(X, Y).$$

Due to floating precision problem, the maximize probability problem is usually turned to maximize log-likelihood problem:

$$\ln P(samples) = \sum_{(X, Y) \in samples} \ln P(X, Y).$$

In order to get robust logistic regression, we usually need to normalize the input value. This code use another trick to avoid floating range problems.

$$-\ln \dfrac{1}{1 + e^{-x}} = \ln(1 + e^{-x}).$$

When $x$ is negative, we use the following formula:

$$\ln (1 + e^{-x}) = \ln(1 + e^x) - x.$$ In the calculation of gradient, we also choose $\dfrac{-e^{-x}}{1 + e^{-x}}$ for positive values, and $\dfrac{-1}{1 + e^x}$ for negative values.

Optimization Demo