\n",
"RangeIndex: 1000 entries, 0 to 999\n",
"Data columns (total 13 columns):\n",
" # Column Non-Null Count Dtype \n",
"--- ------ -------------- ----- \n",
" 0 fage 829 non-null float64\n",
" 1 mage 1000 non-null int64 \n",
" 2 mature 1000 non-null object \n",
" 3 weeks 998 non-null float64\n",
" 4 premie 998 non-null object \n",
" 5 visits 991 non-null float64\n",
" 6 marital 999 non-null object \n",
" 7 gained 973 non-null float64\n",
" 8 weight 1000 non-null float64\n",
" 9 lowbirthweight 1000 non-null object \n",
" 10 gender 1000 non-null object \n",
" 11 habit 999 non-null object \n",
" 12 whitemom 998 non-null object \n",
"dtypes: float64(5), int64(1), object(7)\n",
"memory usage: 101.7+ KB\n"
]
}
],
"source": [
"nc.info()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you review the variable summaries, consider which variables are categorical and which are numerical. For numerical variables, are there outliers? If you aren't sure or want to take a closer look at the data, make a graph.\n",
"\n",
"Consider the possible relationship between a mother's smoking habit and the weight of her baby. Plotting the data is a useful first step because it helps us quickly visualize trends, identify strong associations, and develop research questions."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"
Exercise 2
\n",
" Make a side-by-side boxplot of habit and weight. What does the plot highlight about the relationship between these two variables?\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The box plots show how the medians of the two distributions compare, but we can also compare the means of the distributions using the following function to split the `weight` variable into the `habit` groups, then take the mean of each using `mean()`."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"habit\n",
"nonsmoker 7.144273\n",
"smoker 6.828730\n",
"Name: weight, dtype: float64"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"nc.groupby(['habit'])['weight'].mean()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There is an observed difference, but is this difference statistically significant? In order to answer this question we will conduct a hypothesis test ."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Inference"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"
Exercise 3
\n",
"Check if the conditions necessary for inference are satisfied. Note that you will need to obtain sample sizes to check the conditions. You can compute the group size using the same groupby command above but replacing mean with size.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"
Exercise 4
\n",
"Write the hypotheses for testing if the average weights of babies born to smoking and non-smoking mothers are different.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will now conduct hypothesis tests for testing if the average weights of babies born to smoking and non-smoking mothers are different. For this task, we can use [`statsmodels`](https://www.statsmodels.org/stable/index.html), a Python module that provides classes and functions for the estimation of many different statistical models, as well as for conducting statistical tests, and statistical data exploration."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"n_smoker = 126.0\n",
"mean_smoker = 6.828730158730157\n",
"sd_smoker = 1.380668106117173\n",
"\n",
"n_nonsmoker = 873.0\n",
"mean_nonsmoker = 7.144272623138621\n",
"sd_nonsmoker = 1.5178105512705897\n",
"\n",
"Standard error = 0.13376049190705977\n",
"Test statistic: Z = -2.359010944933654\n",
"p-value = 0.018323715325158647\n",
"reject null hypothesis\n"
]
}
],
"source": [
"import statsmodels.stats.weightstats as st\n",
"\n",
"nc_weightANDsmoker = nc[nc['habit'] == 'smoker']['weight']\n",
"nc_weightANDnonsmoker = nc[nc['habit'] == 'nonsmoker']['weight']\n",
"\n",
"dsw1 = st.DescrStatsW(nc_weightANDsmoker)\n",
"dsw2 = st.DescrStatsW(nc_weightANDnonsmoker)\n",
"cm = st.CompareMeans(dsw1, dsw2)\n",
"\n",
"# calculate number of observations, mean and standard deviation for each group\n",
"n_smoker = dsw1.nobs\n",
"n_nonsmoker = dsw2.nobs\n",
"mean_smoker = dsw1.mean\n",
"mean_nonsmoker = dsw2.mean\n",
"sd_smoker = dsw1.std\n",
"sd_nonsmoker = dsw2.std\n",
"print(f'n_smoker = {n_smoker}')\n",
"print(f'mean_smoker = {mean_smoker}')\n",
"print(f'sd_smoker = {sd_smoker}')\n",
"print()\n",
"print(f'n_nonsmoker = {n_nonsmoker}')\n",
"print(f'mean_nonsmoker = {mean_nonsmoker}')\n",
"print(f'sd_nonsmoker = {sd_nonsmoker}')\n",
"print()\n",
"\n",
"# conduct hypothesis test\n",
"ht = cm.ztest_ind(alternative = 'two-sided', usevar = 'unequal', value = 0)\n",
"\n",
"# calculate and print the standard error, the Z-score, and p-value for the hypothesis test\n",
"se = cm.std_meandiff_separatevar\n",
"testZ = ht[0]\n",
"p_value = ht[1]\n",
"print(f'Standard error = {se}')\n",
"print(f'Test statistic: Z = {testZ}')\n",
"print(f'p-value = {p_value}')\n",
"\n",
"# reject or accept null hypothesis\n",
"if(p_value) < 0.05:\n",
" print('reject null hypothesis')\n",
"else:\n",
" print('accept null hypothesis')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"
Exercise 5
\n",
"Construct a confidence interval for the difference between the weights of babies born to smoking and non-smoking mothers."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## On Your Own\n",
"\n",
"\n",
" - Calculate a 95% confidence interval for the average length of pregnancies (
weeks) and interpret it in context. Note that since you're doing inference on a single population parameter, there is no explanatory variable, so you can omit the x variable from the function.
\n",
" - Calculate a new confidence interval for the same parameter at the 90% confidence level.
\n",
" - Conduct a hypothesis test evaluating whether the average weight gained by younger mothers is different than the average weight gained by mature mothers.
\n",
" - Now, a non-inference task: Determine the age cutoff for younger and mature mothers. Use a method of your choice, and explain how your method works.
\n",
" - Pick a pair of numerical and categorical variables and come up with a research question evaluating the relationship between these variables. Formulate the question in a way that it can be answered using a hypothesis test and/or a confidence interval. Answer your question with Python, report the statistical results, and also provide an explanation in plain language.
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"This lab was adapted by David Akman and Imran Ture from OpenIntro by Andrew Bray and Mine Çetinkaya-Rundel.\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"***"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 4
}