III-34 · Kernel Methods

Change similarity; solve the new boundary

Replace dot products with valid kernels and reconstruct nonlinear decisions from the actual fitted support coefficients.

Part VII · margin laboratory

Fit similarities; trace the actual decision

Hand-authored synthetic labeled observations; no seed or random draws. Edits change the fitted data. The same numerical result drives the plot, equations, ledger and diagnostics.

+ Positive / − Negative · ring: α > 10⁻⁹ · solid f=0 · dashed f=±1. No boundary is drawn for infeasible or failed fits. Nonlinear contours interpolate a 32×32 mesh; training decisions use the actual kernel sum.

Drag a handle; arrows move 0.1, Shift + arrow 0.5. Pointer edits snap to 0.1 within ±5. Numeric entries keep finer precision. Select coincident points to bring their handles forward. Scroll outside handles.

Sum-hinge tradeoffPenalty C0.01 to 20 · Enter to apply

Valid PSD kernel familyKernelKernel gamma0.05 to 2 · Enter to apply

Observation editorSelected pointPoint x-5 to 5 · Enter to applyPoint y-5 to 5 · Enter to applyClass label

Approximate dual fit; inspect KKT and primal–dual gap.

Frame 1 / 1. Editing starts from this displayed frame.

X / y / α shapesBatch rows; individual x and linear w are 2 × 1 columns.: 4 × 2 / 4 × 1 / 4 × 1
StatusApproximate dual fit; inspect KKT and primal–dual gap.: approximate
w / b: Implicit or unavailable / 0
Norm² / half-marginNonlinear norms/margins refer to feature space; zero norm has no distance.: 4.15065 / 0.49084
Primal / dual / gapSum hinge convention; no division by n. Training accuracy is a different metric.: 2.07533 / 2.07533 / 6.789e-7
KKT / equality / bounds residual: 1.146e-7 / 0 / 0
Pair steps / capHard fit is geometric, not iterative. Soft/kernel solutions are approximate.: 5 / 1200

\min_{w,b}\frac12\lVert w\rVert^2+4\sum_i\max(0,1-y_if_i)

f(x)=\sum_i\alpha_i y_i K(x_i,x)+b,\quad\sum_i\alpha_i y_i=0

\lVert w\rVert_{\mathcal H}^2=(\alpha\odot y)^TK(\alpha\odot y),\quad f(x)\ne\text{input distance}

Every training decision and constraint from the active fit
ID / label	f(x)	Decision	y f(x)	ξ = hinge	Signed distance	α	Support / on margin
a / 1	1	1	1	0	Unavailable	1.03766	Yes / Yes
b / 1	1	1	1	1.146e-7	Unavailable	1.03766	Yes / Yes
c / -1	−1	-1	1	0	Unavailable	1.03766	Yes / Yes
d / -1	−1	-1	1	1.146e-7	Unavailable	1.03766	Yes / Yes

Gram K · 4 × 4; entries are similarities, not probabilities
Row / column	a	b	c	d
a	1	0.00034	0.01832	0.01832
b	0.00034	1	0.01832	0.01832
c	0.01832	0.01832	1	0.00034
d	0.01832	0.01832	0.00034	1

approximate. Approximate dual fit; inspect KKT and primal–dual gap. Objective 2.07533, KKT residual 1.146e-7, gap 6.789e-7. Nonlinear score is not input-space distance.

The XOR data cannot be separated by an input-space line. Compare actual linear, polynomial and RBF fits; the Gram matrix, coefficients and contours are reconstructed from the same numerical result.

Ask what a line can express

The opening XOR observations have positive labels at [−1,−1] and [1,1], negative labels at [−1,1] and [1,−1]. Their convex hulls intersect, so a hard input-space line is impossible. Switch to linear: the soft optimum can account for disagreement, but cannot invent nonlinear expressiveness. With the symmetric cloud its normal can be zero, giving ties rather than a separating line.

Switch to polynomial, degree 2, gamma 1 and offset 0. A product of coordinates distinguishes the diagonals. Switch to RBF and alter gamma. The similarity scale changes, the optimizer refits, and the resulting curved zero contour changes accordingly. The laboratory never puts a decorative curve over an unrelated linear model.

Define a valid similarity

A kernel represents an inner product in a feature space:

K(x,z)=\langle\phi(x),\phi(z)\rangle_{\mathcal H}.

For a finite sample, the Gram matrix has n×n shape with Kij=K(xi,xj). It must be symmetric positive semidefinite: for every real column c, cᵀKc≥0. Symmetry alone is insufficient, and kernel entries need not be probabilities or individually nonnegative. PSD is required for the dual’s concavity and for interpreting its quadratic form as a squared feature-space norm.

The laboratory restricts parameters to known valid kernel families; it does not accept an arbitrary user-supplied similarity matrix. The independent oracle checks Gram eigenvalues with NumPy eigvalsh and separately checks literal entries. This tests the implementation; finite-sample PSD alone does not prove a proposed kernel is valid for every possible dataset.

Three kernel families and bounded parameters

The linear kernel K(x,z)=xᵀz gives the original input-space model. The polynomial kernel is

K(x,z)=(\gamma x^Tz+c_0)^d, \quad\gamma>0,\ c_0\ge0,\ d\in\{1,2,3,4\}.

Nonnegative coefficients and an integral degree make this a valid polynomial inner product. The laboratory bounds gamma to [0.05,2] and offset to [0,2]. Degree must be integral; a fractional exponent is rejected even if one particular base would give a finite result. Offset zero gives a homogeneous polynomial; a positive offset permits lower-degree terms.

The radial basis kernel is

K(x,z)=\exp(-\gamma\|x-z\|^2),\qquad\gamma\in[0.05,2].

It gives one on the diagonal and decreases with separation. Increasing gamma makes similarity more local. It is not a Gaussian probability density: there is no normalization constant, and Gram rows do not sum to one. RBF feature space is implicit and generally infinite-dimensional. The code computes finite similarities and finite kernel sums; it does not materialize infinite coordinate vectors.

Derive the kernel decision from the dual

The representer form of the fitted feature-space normal is w=Σαi yi φ(xi). Thus the same bounded soft dual uses only K, and prediction is

f(x)=\sum_i\alpha_i y_i K(x_i,x)+b,\quad \hat y=\operatorname{sign}f(x),\quad y^T\alpha=0,\quad0\le\alpha_i\le C.

The α column has n×1 shape. X still stores n×2 source rows, but the implicit feature-space normal can have a different or infinite dimension. There is no source-space weight vector for a nonlinear fit. Its squared norm is

\|w\|_{\mathcal H}^2=(\alpha\odot y)^TK(\alpha\odot y).

The regularizer and primal–dual diagnostics use this feature norm. Dividing f(x) by the feature norm, f(x)/‖w‖𝓗 (the square root of the displayed norm squared), gives a feature-space signed distance when the normal is nonzero, but generally not Euclidean distance from x to a curved input-space boundary. The laboratory deliberately leaves source-space signed distance unavailable for nonlinear kernels. Its f=±1 contours represent functional margin levels; their visual spacing in the input plane need not be constant.

Positive α means a support coefficient, while $0<\alpha<C$ and α=C have the KKT interpretations from III-33. Location alone does not determine a coefficient. On-margin flags remain separate and the actual Gram matrix stays visible even if fitting diagnostics fail.

Work a polynomial feature map literally

For degree 2, gamma 1, offset 0 in two dimensions,

\phi(x)=\begin{bmatrix}x_1^2\\\sqrt2 x_1x_2\\x_2^2\end{bmatrix},\quad \phi(x)^T\phi(z)=(x^Tz)^2.

For the XOR rows in opening order, within-class diagonally opposite vectors have dot product −2; self dot products are 2; cross-class adjacent vectors have dot product zero. Squaring gives

K=\begin{bmatrix}4&4&0&0\\4&4&0&0\\0&0&4&4\\0&0&4&4\end{bmatrix}.

For C≥1/8, one symmetric optimum has all α=1/8 and b=0, yielding f(x)=x1x2, feature norm squared 1/2 and primal=dual=1/4. The feature normal is [0,1/√2,0]ᵀ. Its zero set consists of both coordinate axes in the input plane. Coefficients are not unique because some feature rows coincide; a deterministic pair solver may place the same total class mass on fewer observations. Compare reconstructed decisions and objective rather than demanding these particular coefficients from every solver.

For x=[1,0], z=[0,1], the linear entry is zero; the degree-2 polynomial with offset 1 is one; RBF gamma 1 is exp(−2). Diagonal RBF entries are one. These are hand-defined expectations, not outputs generated by the browser.

For the XOR RBF kernel, write a=exp(−4), so cross-class entries are a and opposite same-class entries a². A symmetric optimum with inactive upper bounds has α=1/(1−a)², b=0 and primal=dual=2/(1−a)². C=4 is above that coefficient. This is a useful analytic anchor for the owner to compare against the actual optimizer.

Execute the actual reference and inspect the mesh

The canonical reference accepts mode: kernel and a complete valid kernel configuration, then constructs a NumPy Gram array and performs bounded pair-coordinate optimization. It reconstructs training decisions, feature norm, hinge, objective, equality and KKT residuals. The oracle independently enumerates dual active sets through linear algebra and checks eigenvalues, entry identities and decisions at additional query points.

code/python/p07/svm.py

"""Canonical bounded two-feature SVM, Python + NumPy. Owner executes checks.

X stores n×2 rows; y/alpha are serialized columns. No randomness. Hard margin
uses closest convex hulls; soft/kernel uses exact feasible pair dual steps.
"""
import json
from fractions import Fraction
import math
import sys
import numpy as np

EPS = 1e-9
KKT_TOL = 2e-5
GAP_TOL = 1e-5
MAX_STEPS = 1200


def real(v, lo=-math.inf, hi=math.inf):
    if isinstance(v, (bool, np.bool_)) or not isinstance(v, (int, float, np.integer, np.floating)):
        raise ValueError("Expected a finite real number, not a Boolean.")
    try:
        x = float(v)
    except (OverflowError, ValueError) as error:
        raise ValueError("Number cannot be represented in binary64.") from error
    if not math.isfinite(x) or not lo <= x <= hi:
        raise ValueError("Number outside the finite supported range.")
    return x


def finite(v):
    if not math.isfinite(v):
        raise ValueError("Computed arithmetic is not representable.")
    return v


def mul(a, b):
    v = finite(a * b)
    if a != 0 and b != 0 and v == 0:
        raise ValueError("Nonzero product underflows binary64.")
    return v


def dot(a, b):
    return finite(mul(a[0], b[0]) + mul(a[1], b[1]))


def sub(a, b):
    return [finite(a[0] - b[0]), finite(a[1] - b[1])]


def validate(s):
    if not isinstance(s, dict) or s.get("mode") not in ("hard", "soft", "kernel"):
        raise ValueError("Choose hard, soft or kernel mode.")
    points = s.get("points")
    if not isinstance(points, list) or not 2 <= len(points) <= 12:
        raise ValueError("Use 2–12 dense observations.")
    ids = set()
    for p in points:
        if not isinstance(p, dict) or not isinstance(p.get("id"), str) or not p["id"].strip() or p["id"] in ids:
            raise ValueError("Observations require unique nonempty IDs.")
        ids.add(p["id"])
        if not isinstance(p.get("x"), (list, tuple)) or len(p["x"]) != 2:
            raise ValueError("Each observation needs exactly two coordinates.")
        for x in p["x"]:
            real(x, -5, 5)
        if real(p.get("y")) not in (-1, 1):
            raise ValueError("Labels must be −1 or +1.")
    if len({p["y"] for p in points}) != 2:
        raise ValueError("Both labels are required.")
    real(s.get("C"), .01, 20)
    k = s.get("kernel")
    if not isinstance(k, dict) or k.get("kind") not in ("linear", "polynomial", "rbf"):
        raise ValueError("Unknown kernel.")
    real(k.get("gamma"), .05, 2)
    real(k.get("coef0"), 0, 2)
    d = real(k.get("degree"), 1, 4)
    if not d.is_integer():
        raise ValueError("Polynomial degree must be integral.")
    return dict(s, points=[dict(id=p["id"], x=[real(x) for x in p["x"]], y=real(p["y"])) for p in points],
                C=real(s["C"]), kernel=dict(k, gamma=real(k["gamma"]), coef0=real(k["coef0"]), degree=int(d)))


def kernel_value(a, b, k):
    """Validate both dense query vectors and all parameters before family dispatch."""
    if not isinstance(a, (list, tuple)) or not isinstance(b, (list, tuple)) or len(a) != 2 or len(b) != 2:
        raise ValueError("Kernel vectors need two dense coordinates.")
    a, b = [real(v, -6, 6) for v in a], [real(v, -6, 6) for v in b]
    if not isinstance(k, dict) or k.get("kind") not in ("linear", "polynomial", "rbf"):
        raise ValueError("Unknown kernel.")
    gamma, coef0, degree = real(k.get("gamma"), .05, 2), real(k.get("coef0"), 0, 2), real(k.get("degree"), 1, 4)
    if not degree.is_integer():
        raise ValueError("Polynomial degree must be integral.")
    k = dict(k, gamma=gamma, coef0=coef0, degree=int(degree))
    if k["kind"] == "linear":
        return dot(a, b)
    if k["kind"] == "polynomial":
        base = finite(mul(k["gamma"], dot(a, b)) + k["coef0"])
        result = 1.0
        for _ in range(k["degree"]):
            result = mul(result, base)
        return result
    delta = sub(a, b)
    value = math.exp(-mul(k["gamma"], dot(delta, delta)))
    if value == 0:
        raise ValueError("RBF similarity underflows binary64.")
    return finite(value)


def hinge(margin):
    m = real(margin)
    return {"loss": max(0, finite(1 - m)), "subgradient": -1 if m < 1 else 0}


def orientation(a, b, c):
    """Filtered sign, with exact fallback from original binary64 coordinates."""
    ux, uy = b[0] - a[0], b[1] - a[1]
    vx, vy = c[0] - a[0], c[1] - a[1]
    left, right = ux * vy, uy * vx
    determinant = left - right
    error = 8 * sys.float_info.epsilon * (abs(left) + abs(right))
    if abs(left) >= sys.float_info.min and abs(right) >= sys.float_info.min and abs(determinant) > error:
        return 1 if determinant > 0 else -1
    ax, ay, bx, by, cx, cy = (Fraction.from_float(float(v)) for p in (a, b, c) for v in p)
    exact = (bx - ax) * (cy - ay) - (by - ay) * (cx - ax)
    return (exact > 0) - (exact < 0)


def hull(points, label):
    indices = sorted((i for i, p in enumerate(points) if p["y"] == label), key=lambda i: tuple(points[i]["x"]))
    unique = []
    for i in indices:
        if not unique or points[i]["x"] != points[unique[-1]]["x"]:
            unique.append(i)
    def chain(seq):
        out = []
        for i in seq:
            while len(out) >= 2 and orientation(points[out[-2]]["x"], points[out[-1]]["x"], points[i]["x"]) <= 0:
                out.pop()
            out.append(i)
        return out
    return unique if len(unique) <= 2 else chain(unique)[:-1] + chain(unique[::-1])[:-1]


def edges(h):
    return [(i, h[(j + 1) % len(h)]) for j, i in enumerate(h)]


def inside(x, h, points):
    return len(h) >= 3 and all(orientation(points[i]["x"], points[j]["x"], x) >= 0 for i, j in edges(h))


def on_segment(x, a, b):
    return orientation(a, b, x) == 0 and all(min(a[j], b[j]) <= x[j] <= max(a[j], b[j]) for j in range(2))


def intersects(a, b, c, d):
    u, v = orientation(a, b, c), orientation(a, b, d)
    r, t = orientation(c, d, a), orientation(c, d, b)
    return ((u < 0 < v or v < 0 < u) and (r < 0 < t or t < 0 < r)) or any((on_segment(c, a, b), on_segment(d, a, b), on_segment(a, c, d), on_segment(b, c, d)))


def hard_fit(points):
    neg, pos = hull(points, -1), hull(points, 1)
    if any(inside(points[i]["x"], pos, points) for i in neg) or any(inside(points[i]["x"], neg, points) for i in pos):
        return None
    best = None
    for ni, nj in edges(neg):
        for pi, pj in edges(pos):
            a, b, c, d = [points[i]["x"] for i in (ni, nj, pi, pj)]
            if intersects(a, b, c, d):
                return None
            for source, i, j in ((ni, pi, pj), (nj, pi, pj), (pi, ni, nj), (pj, ni, nj)):
                x, u, v = points[source]["x"], points[i]["x"], points[j]["x"]
                delta = sub(v, u)
                length = dot(delta, delta)
                t = max(0, min(1, dot(sub(x, u), delta) / length)) if length else 0
                q = [finite(u[z] + mul(t, delta[z])) for z in range(2)]
                diff = sub(q, x)
                distance2 = dot(diff, diff)
                if best is None or distance2 < best[0]:
                    best = (distance2, source, i, j, t, q)
    distance2, source, i, j, t, q = best
    if distance2 == 0:
        raise ValueError("Closest hull distance is numerically unresolved.")
    mass = finite(2 / distance2)
    y = points[source]["y"]
    direction = sub(points[source]["x"], q) if y == 1 else sub(q, points[source]["x"])
    w = [mul(mass, d) for d in direction]
    midpoint = [finite((points[source]["x"][z] + q[z]) / 2) for z in range(2)]
    alpha = [0.0] * len(points)
    alpha[source] += mass
    alpha[i] += mul(mass, 1 - t)
    alpha[j] += mul(mass, t)
    return alpha, -dot(w, midpoint), w


def soft_fit(points, K, C):
    n = len(points)
    y = [p["y"] for p in points]
    alpha, g = [0.0] * n, [0.0] * n
    iterations = 0
    for step in range(MAX_STEPS):
        best = None
        for i in range(n):
            for j in range(i + 1, n):
                di, dj = y[i], -y[j]
                li, ui = (-alpha[i], C - alpha[i]) if di == 1 else (alpha[i] - C, alpha[i])
                lj, uj = (-alpha[j], C - alpha[j]) if dj == 1 else (alpha[j] - C, alpha[j])
                lo, hi = max(li, lj), min(ui, uj)
                if hi <= lo:
                    continue
                slope = finite(y[i] - y[j] - (g[i] - g[j]))
                curvature = finite(K[i][i] + K[j][j] - 2 * K[i][j])
                if curvature < -1e-10:
                    raise ValueError("Kernel curvature contradicts PSD within the numerical allowance.")
                candidates = [lo, hi] if curvature <= 0 else [max(lo, min(hi, slope / curvature))]
                for t in candidates:
                    gain = finite(mul(slope, t) - .5 * mul(curvature, mul(t, t)))
                    if gain > 1e-12 and (best is None or gain > best[0]):
                        best = (gain, i, j, t)
        if best is None:
            break
        _, i, j, t = best
        alpha[i] = finite(alpha[i] + y[i] * t)
        alpha[j] = finite(alpha[j] - y[j] * t)
        for z in range(n):
            g[z] = finite(g[z] + mul(t, K[z][i] - K[z][j]))
        iterations = step + 1
    # Reconstruct instead of letting accumulated gradient rounding own diagnostics.
    g = [finite(sum(mul(alpha[j] * y[j], K[i][j]) for j in range(n))) for i in range(n)]
    free = [y[i] - g[i] for i in range(n) if EPS < alpha[i] < C - EPS]
    if free:
        b = finite(sum(free) / len(free))
    else:
        lower, upper = -math.inf, math.inf
        for i in range(n):
            target = y[i] - g[i]
            if (y[i] == 1 and alpha[i] <= EPS) or (y[i] == -1 and alpha[i] >= C - EPS):
                lower = max(lower, target)
            else:
                upper = min(upper, target)
        b = finite((lower + upper) / 2)
    return alpha, b, iterations


def svm(request):
    s = validate(request)
    points, C = s["points"], s["C"]
    n = len(points)
    k = s["kernel"] if s["mode"] == "kernel" else dict(s["kernel"], kind="linear")
    K = np.array([[kernel_value(a["x"], b["x"], k) for b in points] for a in points], dtype=float).tolist()
    if s["mode"] == "hard":
        fitted = hard_fit(points)
        if fitted is None:
            return dict(status="infeasible", reason="Class convex hulls intersect; hard-margin constraints are infeasible.", gram=K, rows=[], alpha=[], w=None, b=None, norm2=None, halfMargin=None, primal=None, dual=None, gap=None, kkt=None, equality=None, bounds=None, iterations=0)
        alpha, b, w = fitted
        iterations = 0
    else:
        alpha, b, iterations = soft_fit(points, K, C)
        w = [finite(sum(mul(alpha[i] * p["y"], p["x"][z]) for i, p in enumerate(points))) for z in range(2)] if k["kind"] == "linear" else None
    scores = [finite(sum(mul(alpha[j] * points[j]["y"], K[i][j]) for j in range(n)) + b) for i in range(n)]
    norm2 = finite(sum(mul(alpha[i] * points[i]["y"], finite(scores[i] - b)) for i in range(n)))
    if norm2 < 0:
        raise ValueError("Computed feature norm is negative; numerical fit unavailable.")
    norm = math.sqrt(norm2)
    rows, residual = [], 0.0
    for i, p in enumerate(points):
        margin = mul(p["y"], scores[i])
        xi = max(0, finite(1 - margin))
        bound = s["mode"] != "hard" and alpha[i] >= C - EPS
        r = max(0, 1 - margin) if alpha[i] <= EPS else max(0, margin - 1) if bound else abs(margin - 1)
        residual = max(residual, r)
        rows.append(dict(id=p["id"], score=scores[i], decision=1 if scores[i] > 0 else -1 if scores[i] < 0 else 0, margin=margin, slack=xi, hinge=xi, signedDistance=finite(scores[i] / norm) if w is not None and norm > 0 else None, support=alpha[i] > EPS, onMargin=abs(margin - 1) <= KKT_TOL, alpha=alpha[i]))
    primal = finite(.5 * norm2 + (0 if s["mode"] == "hard" else C * sum(r["hinge"] for r in rows)))
    dual = finite(sum(alpha) - .5 * norm2)
    gap = finite(primal - dual)
    equality = abs(finite(sum(alpha[i] * p["y"] for i, p in enumerate(points))))
    bounds = max([0] + [-a for a in alpha] + ([] if s["mode"] == "hard" else [a - C for a in alpha]))
    feasible = s["mode"] != "hard" or max(r["slack"] for r in rows) <= KKT_TOL
    good = feasible and residual <= KKT_TOL and abs(gap) <= GAP_TOL * (1 + primal) and equality <= 1e-7 and bounds <= EPS
    return dict(status=("solved" if s["mode"] == "hard" else "approximate") if good else "failed", reason="Hard fit certified within displayed diagnostics." if good and s["mode"] == "hard" else "Approximate dual fit; inspect KKT and primal–dual gap." if good else "Diagnostic failure or iteration cap; no fitted boundary is displayed.", gram=K, rows=rows, alpha=alpha, w=w, b=b, norm2=norm2, halfMargin=finite(1 / norm) if norm > 0 else None, primal=primal, dual=dual, gap=gap, kkt=residual, equality=equality, bounds=bounds, iterations=iterations)


def decision(request, result, x):
    """Actual kernel decision; never input-space distance for nonlinear kernels."""
    s = validate(request)
    if result.get("status") not in ("solved", "approximate") or result.get("b") is None or len(result.get("alpha", [])) != len(s["points"]):
        raise ValueError("A successful fit is required for a decision.")
    k = s["kernel"] if s["mode"] == "kernel" else dict(s["kernel"], kind="linear")
    return finite(sum(mul(real(a) * p["y"], kernel_value(p["x"], x, k)) for a, p in zip(result["alpha"], s["points"])) + real(result["b"]))


DEFAULT = dict(mode="hard", C=1, kernel=dict(kind="linear", gamma=1, coef0=1, degree=2), points=[dict(id="a", x=[-1, 0], y=-1), dict(id="b", x=[1, 0], y=1)])

if __name__ == "__main__":
    try:
        raw = sys.stdin.read()
        print(json.dumps(svm(json.loads(raw) if raw.strip() else DEFAULT), allow_nan=False))
    except (ValueError, TypeError, KeyError, OverflowError) as error:
        print(json.dumps({"error": str(error)}), file=sys.stderr)
        sys.exit(2)

The input plane is sampled on a 32×32 mesh. Cell colors use actual kernel decisions at cell centers. Zero and ±1 contour segments interpolate corner scores; they are a finite-resolution visualization of the solved function. Ambiguous saddle cells use deterministic edge-order pairing. Training rows and any tested queries use the full kernel sum rather than that interpolation. A coarse contour can miss small islands; it is not evidence of exact source-space geometry.

Select Show diagnostic failure to load the three-row polynomial case with C=1, gamma=1, offset=1 and degree=4. Its valid Gram matrix remains inspectable, but the fixed gain cutoff leaves a KKT residual above the required bound. The laboratory reports failed and draws no fitted contours. This is optimization unavailability, not hard-margin infeasibility; reset restores the opening experiment.

Replay C and then edit a sample. The displayed coefficients and boundary are recomputed from the selected historical input frame. Reset restores RBF gamma 1, C=4 and XOR rows. Keyboard, numeric and touch controls use the same update path and reject invalid values visibly.

Common mistakes and numerical limits

A nonlinear score is not an input-space distance or calibrated probability. More localized RBF similarity can fit small-scale training patterns and still generalize poorly. Large polynomial entries can worsen conditioning even with valid bounded parameters. PSD roundoff is diagnostic: significantly negative pair curvature is rejected; a nonpositive curvature within the fixed 10⁻¹⁰ numerical allowance uses feasible endpoints and still must pass global KKT/gap diagnostics.

Five graded investigations

Connections and summary

Dot products, PSD matrices and convex duality explain the kernel trick. You can read Gram entries, reconstruct decisions and distinguish feature-space norms from source-space distance. The model family, C and gamma remain hyperparameters: III-35 will explain how to select such settings using validation evidence rather than training fit alone.