diff --git a/chapter_appendix-mathematics-for-deep-learning/single-variable-calculus.md b/chapter_appendix-mathematics-for-deep-learning/single-variable-calculus.md index a6d8438b6d..62727a4c0c 100644 --- a/chapter_appendix-mathematics-for-deep-learning/single-variable-calculus.md +++ b/chapter_appendix-mathematics-for-deep-learning/single-variable-calculus.md @@ -251,7 +251,7 @@ Where each line has used the following rules: Two things should be clear after doing this example: -1. Any function we can write down using sums, products, constants, powers, exponentials, and logarithms can have its derivate computed mechanically by following these rules. +1. Any function we can write down using sums, products, constants, powers, exponentials, and logarithms can have its derivative computed mechanically by following these rules. 2. Having a human follow these rules can be tedious and error prone! Thankfully, these two facts together hint towards a way forward: this is a perfect candidate for mechanization! Indeed backpropagation, which we will revisit later in this section, is exactly that. diff --git a/chapter_appendix-mathematics-for-deep-learning/statistics.md b/chapter_appendix-mathematics-for-deep-learning/statistics.md index a37d085231..db0b432ee9 100644 --- a/chapter_appendix-mathematics-for-deep-learning/statistics.md +++ b/chapter_appendix-mathematics-for-deep-learning/statistics.md @@ -122,7 +122,7 @@ Perhaps the simplest metric used to evaluate estimators is the *mean squared err $$\textrm{MSE} (\hat{\theta}_n, \theta) = E[(\hat{\theta}_n - \theta)^2].$$ :eqlabel:`eq_mse_est` -This allows us to quantify the average squared deviation from the true value. MSE is always non-negative. If you have read :numref:`sec_linear_regression`, you will recognize it as the most commonly used regression loss function. As a measure to evaluate an estimator, the closer its value to zero, the closer the estimator is close to the true parameter $\theta$. +This allows us to quantify the average squared deviation from the true value. MSE is always non-negative. If you have read :numref:`sec_linear_regression`, you will recognize it as the most commonly used regression loss function. As a measure to evaluate an estimator, the closer its value is to zero, the closer the estimator is to the true parameter $\theta$. ### Statistical Bias @@ -148,7 +148,7 @@ Second, let's measure the randomness in the estimator. Recall from :numref:`sec $$\sigma_{\hat{\theta}_n} = \sqrt{\textrm{Var} (\hat{\theta}_n )} = \sqrt{E[(\hat{\theta}_n - E(\hat{\theta}_n))^2]}.$$ :eqlabel:`eq_var_est` -It is important to compare :eqref:`eq_var_est` to :eqref:`eq_mse_est`. In this equation we do not compare to the true population value $\theta$, but instead to $E(\hat{\theta}_n)$, the expected sample mean. Thus we are not measuring how far the estimator tends to be from the true value, but instead we measuring the fluctuation of the estimator itself. +It is important to compare :eqref:`eq_var_est` to :eqref:`eq_mse_est`. In this equation we do not compare to the true population value $\theta$, but instead to $E(\hat{\theta}_n)$, the expected sample mean. Thus we are not measuring how far the estimator tends to be from the true value, but instead we are measuring the fluctuation of the estimator itself. ### The Bias-Variance Trade-off @@ -166,7 +166,7 @@ $$ \end{aligned} $$ -We refer the above formula as *bias-variance trade-off*. The mean squared error can be divided into three sources of error: the error from high bias, the error from high variance and the irreducible error. The bias error is commonly seen in a simple model (such as a linear regression model), which cannot extract high dimensional relations between the features and the outputs. If a model suffers from high bias error, we often say it is *underfitting* or lack of *flexibilty* as introduced in (:numref:`sec_generalization_basics`). The high variance usually results from a too complex model, which overfits the training data. As a result, an *overfitting* model is sensitive to small fluctuations in the data. If a model suffers from high variance, we often say it is *overfitting* and lack of *generalization* as introduced in (:numref:`sec_generalization_basics`). The irreducible error is the result from noise in the $\theta$ itself. +We refer the above formula as *bias-variance trade-off*. The mean squared error can be divided into three sources of error: the error from high bias, the error from high variance and the irreducible error. The bias error is commonly seen in a simple model (such as a linear regression model), which cannot extract high dimensional relations between the features and the outputs. If a model suffers from high bias error, we often say it is *underfitting* or lack of *flexibility* as introduced in (:numref:`sec_generalization_basics`). The high variance usually results from a too complex model, which overfits the training data. As a result, an *overfitting* model is sensitive to small fluctuations in the data. If a model suffers from high variance, we often say it is *overfitting* and lack of *generalization* as introduced in (:numref:`sec_generalization_basics`). The irreducible error is the result from noise in the $\theta$ itself. ### Evaluating Estimators in Code @@ -268,7 +268,7 @@ tf.square(tf.math.reduce_std(samples)) + tf.square(bias) ## Conducting Hypothesis Tests -The most commonly encountered topic in statistical inference is hypothesis testing. While hypothesis testing was popularized in the early 20th century, the first use can be traced back to John Arbuthnot in the 1700s. John tracked 80-year birth records in London and concluded that more men were born than women each year. Following that, the modern significance testing is the intelligence heritage by Karl Pearson who invented $p$-value and Pearson's chi-squared test, William Gosset who is the father of Student's t-distribution, and Ronald Fisher who initialed the null hypothesis and the significance test. +The most commonly encountered topic in statistical inference is hypothesis testing. While hypothesis testing was popularized in the early $20^{th}$ century, the first use can be traced back to John Arbuthnot in the 1700s. John tracked 80-year birth records in London and concluded that more men were born than women each year. Following that, the modern significance testing is the intelligence heritage by Karl Pearson who invented $p$-value and Pearson's chi-squared test, William Gosset who is the father of Student's t-distribution, and Ronald Fisher who initialed the null hypothesis and the significance test. A *hypothesis test* is a way of evaluating some evidence against the default statement about a population. We refer the default statement as the *null hypothesis* $H_0$, which we try to reject using the observed data. Here, we use $H_0$ as a starting point for the statistical significance testing. The *alternative hypothesis* $H_A$ (or $H_1$) is a statement that is contrary to the null hypothesis. A null hypothesis is often stated in a declarative form which posits a relationship between variables. It should reflect the brief as explicit as possible, and be testable by statistics theory. diff --git a/chapter_recommender-systems/neumf.md b/chapter_recommender-systems/neumf.md index 05d0b5a543..bbed759c1c 100644 --- a/chapter_recommender-systems/neumf.md +++ b/chapter_recommender-systems/neumf.md @@ -224,7 +224,7 @@ train_iter = gluon.data.DataLoader( True, last_batch="rollover", num_workers=d2l.get_dataloader_workers()) ``` -We then create and initialize the model. we use a three-layer MLP with constant hidden size 10. +We then create and initialize the model. We use a three-layer MLP with constant hidden size 10. ```{.python .input n=8} #@tab mxnet