20240520

BlitherBoom812 · May 20, 2024 · c28d377 · c28d377
1 parent 1f5b336
commit c28d377
Show file tree

Hide file tree

Showing 17 changed files with 85 additions and 0 deletions.
diff --git a/source/_posts/AI.md b/source/_posts/AI.md
@@ -417,3 +417,87 @@ $$
 p(x_1,...,x_K)=p(x_K|x_1,...,x_{K-1})\cdots p(x_2|x_1)p(x_1)
 $$
 
+#### Variable Elimination
+
+用于计算概率的边缘分布
+
+![1716191759894](../images/AI/1716191759894.png)
+
+一般而言是 NP-hard 问题。
+
+对于 Markov chain，复杂度为 $O(nk^2)$；对于一般的图，$O(k^{n-1})$；如果确定每个节点的父节点数不超过 m，则复杂度为 $O(nk^{m-1})$
+
+#### Message Passing
+
+Reuse the computation from $P(Y|E=e)$ when calcuating another probability $P(Y_1|E_1=e_1)$
+
+![1716192135361](../images/AI/1716192135361.png)
+
+![1716192175996](../images/AI/1716192175996.png)
+
+“$\propto$” 意味着只需要知道概率的相对值就够了，因为可以通过归一化算出最终的概率值。
+
+MAP 需要求概率分布的最大值。
+
+sum 与 max 同为聚合操作，因此同样满足分配律，只需要对应替换就可以得到第二种 Message Passing：
+
+![1716192534058](../images/AI/1716192534058.png)
+
+
+![1716192519850](../images/AI/1716192519850.png)
+
+![1716193180893](../images/AI/1716193180893.png)
+
+### Bayes Approach
+
+![1716193473766](../images/AI/1716193473766.png)
+
+### MLE method
+
+如果概率模型的参数知道，称为概率；不知道，称为统计推断。
+
+估计高斯分布的参数：
+
+方差是有偏估计，所以一般× $1/(n-1)$。
+
+![1716194228666](../images/AI/1716194228666.png)
+
+### Bayes Decision Rule
+
+![1716194320872](../images/AI/1716194320872.png)
+
+对于回归问题，可以采用高斯噪声假设：
+
+![1716194583833](../images/AI/1716194583833.png)
+
+这样就得到了最小二乘估计。
+
+MLE 是先验概率相等的 MAP。
+
+放在机器学习中，MAP 可以定义为：模型 = 数据 + 先验。
+
+![1716194633162](../images/AI/1716194633162.png)
+
+先验信息在机器学习中体现为正则化：
+
+2 范数正则化就是在认为模型参数服从高斯分布的先验假设情况下，利用 MAP 准则来估计参数。
+
+这也就是为什么正则化倾向于避免过拟合：高斯分布先验希望模型参数足够简单。
+
+![1716194757145](../images/AI/1716194757145.png)
+
+### Bayesian Model Averaging
+
+![1716195002371](../images/AI/1716195002371.png)
+
+意义：模型集成。
+
+### Discriminative Models
+
+上面的理论足够解释判别式模型的原理了。
+
+### Generative Models
+
+![1716195398036](../images/AI/1716195398036.png)
+
+![1716195442145](../images/AI/1716195442145.png)
diff --git a/source/_posts/Antenna.md b/source/_posts/Antenna.md
@@ -663,3 +663,4 @@ The E-pattern is in shadow.
 
 ![1715929531751](../images/Antenna/1715929531751.png)
 
+![1715929546494](../images/Antenna/1715929546494.png)
diff --git a/source/images/AI/1716191759894.png b/source/images/AI/1716191759894.png
diff --git a/source/images/AI/1716192135361.png b/source/images/AI/1716192135361.png
diff --git a/source/images/AI/1716192175996.png b/source/images/AI/1716192175996.png
diff --git a/source/images/AI/1716192519850.png b/source/images/AI/1716192519850.png
diff --git a/source/images/AI/1716192534058.png b/source/images/AI/1716192534058.png
diff --git a/source/images/AI/1716193180893.png b/source/images/AI/1716193180893.png
diff --git a/source/images/AI/1716193473766.png b/source/images/AI/1716193473766.png
diff --git a/source/images/AI/1716194228666.png b/source/images/AI/1716194228666.png
diff --git a/source/images/AI/1716194320872.png b/source/images/AI/1716194320872.png
diff --git a/source/images/AI/1716194583833.png b/source/images/AI/1716194583833.png
diff --git a/source/images/AI/1716194633162.png b/source/images/AI/1716194633162.png
diff --git a/source/images/AI/1716194757145.png b/source/images/AI/1716194757145.png
diff --git a/source/images/AI/1716195002371.png b/source/images/AI/1716195002371.png
diff --git a/source/images/AI/1716195398036.png b/source/images/AI/1716195398036.png
diff --git a/source/images/AI/1716195442145.png b/source/images/AI/1716195442145.png