{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "intro",
      "metadata": {},
      "source": [
        "# XORとAttentionを小さな配列で試す\n",
        "\n",
        "依存ライブラリ、共通準備、XORまたはAttention、結果を保存の順です。片方だけでも保存できます。記事は2026年10月20日JSTにサーバーCPUで取得した結果で、ColabやGPUでの実測ではありません。\n",
        "\n",
        "- XOR: https://ai.technology-developer.com/notes/ai-evolution-xor/\n",
        "- Attention: https://ai.technology-developer.com/notes/ai-evolution-attention/\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "install-heading",
      "metadata": {},
      "source": [
        "## 依存ライブラリ\n",
        "\n",
        "初回はライブラリの取得に通信します。APIキーや学習済みLLMは使いません。\n"
      ]
    },
    {
      "cell_type": "code",
      "id": "install",
      "metadata": {},
      "source": [
        "%pip install -q numpy==2.4.6 scipy==1.16.2 scikit-learn==1.7.2\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "setup-heading",
      "metadata": {},
      "source": [
        "## 共通準備\n"
      ]
    },
    {
      "cell_type": "code",
      "id": "setup",
      "metadata": {},
      "source": [
        "from pathlib import Path\n",
        "from datetime import datetime, timezone\n",
        "import json, platform, warnings\n",
        "import numpy as np\n",
        "import scipy, sklearn\n",
        "from sklearn.linear_model import Perceptron\n",
        "from sklearn.neural_network import MLPClassifier\n",
        "record={'location':'notebook CPU examples','python':platform.python_version(),'versions':{'numpy':np.__version__,'scipy':scipy.__version__,'sklearn':sklearn.__version__},'seed':42}\n",
        "print(record)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "xor-heading",
      "metadata": {},
      "source": [
        "## XOR\n",
        "\n",
        "学習した4点そのものへの適合を確認します。未知のデータの精度を測る比較ではありません。\n"
      ]
    },
    {
      "cell_type": "code",
      "id": "xor",
      "metadata": {},
      "source": [
        "X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=float)\n",
        "y = np.array([0, 1, 1, 0])\n",
        "configurations = [\n",
        "    ('perceptron', Perceptron(max_iter=1000, tol=None, random_state=42, shuffle=False)),\n",
        "    ('mlp-tanh', MLPClassifier(hidden_layer_sizes=(8,), activation='tanh', solver='lbfgs', max_iter=2000, random_state=42, tol=1e-8, alpha=0.0001)),\n",
        "    ('mlp-identity', MLPClassifier(hidden_layer_sizes=(8,), activation='identity', solver='lbfgs', max_iter=2000, random_state=42, tol=1e-8, alpha=0.0001)),\n",
        "]\n",
        "fitted=[]\n",
        "for name, model in configurations:\n",
        "    with warnings.catch_warnings(record=True) as captured:\n",
        "        warnings.simplefilter('always')\n",
        "        model.fit(X, y)\n",
        "    predicted=model.predict(X)\n",
        "    fitted.append({'name':name,'parameters':model.get_params(),'predicted':predicted.tolist(),'correct':int(np.sum(predicted==y)),\n",
        "                   'nIter':int(model.n_iter_),'warnings':[str(w.message) for w in captured]})\n",
        "record['xor']={'inputs':X.tolist(),'expected':y.tolist(),'evaluatedOnTrainingPoints':True,'runs':fitted}\n",
        "print(json.dumps(record['xor'],ensure_ascii=False,indent=2))\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "attention-heading",
      "metadata": {},
      "source": [
        "## Attention\n",
        "\n",
        "3位置×2次元の手で用意した配列です。実モデルの注目先を測るものではありません。Qのみ・Vのみの変更はそれぞれ基準から独立して行います。\n"
      ]
    },
    {
      "cell_type": "code",
      "id": "attention",
      "metadata": {},
      "source": [
        "Q = np.array([[1., 0.], [0., 1.], [1., 1.]])\n",
        "K = Q.copy()\n",
        "V = np.array([[1., 0.], [0., 1.], [1., 1.]])\n",
        "attention=[]\n",
        "for name,q,v in [('baseline',Q,V),('query-first-row-changed',np.array([[0.,1.],[0.,1.],[1.,1.]]),V),('value-first-row-changed',Q,np.array([[2.,0.],[0.,1.],[1.,1.]]))]:\n",
        "    scores=q @ K.T / np.sqrt(2)\n",
        "    weights=np.exp(scores-scores.max(axis=1,keepdims=True))\n",
        "    weights/=weights.sum(axis=1,keepdims=True)\n",
        "    output=weights @ v\n",
        "    attention.append({'name':name,'Q':q.tolist(),'K':K.tolist(),'V':v.tolist(),'scores':scores.tolist(),'weights':weights.tolist(),'rowSums':weights.sum(axis=1).tolist(),'output':output.tolist()})\n",
        "record['attention']={'manuallyChosenQKV':True,'causalMask':False,'trainedLanguageModelUsed':False,'runs':attention}\n",
        "print(json.dumps(record['attention'],ensure_ascii=False,indent=2))\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "save-heading",
      "metadata": {},
      "source": [
        "## 結果を保存\n"
      ]
    },
    {
      "cell_type": "code",
      "id": "save",
      "metadata": {},
      "source": [
        "record['completedAt']=datetime.now(timezone.utc).isoformat()\n",
        "Path('foundation-results-local.json').write_text(json.dumps(record,ensure_ascii=False,indent=2,allow_nan=False)+'\\n')\n",
        "print('foundation-results-local.json に実行結果を保存しました。')\n"
      ],
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.11"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}