MODULE II: DESCRIPTIVE STATISTICS, CROSS-TABULATION & DATA VISUALIZATION
1. DESCRIPTIVE STATISTICS IN STATISTICAL SOFTWARE ENVIRONMENTS
Before estimating econometric equations or conducting confirmatory hypothesis tests, empirical researchers must thoroughly explore the fundamental distributional properties of their dataset. Descriptive Statistics condense massive raw data matrices into manageable numerical metrics that reveal the central location, variability, shape, and potential anomalies of commercial indicators.
Measures of Central Tendency
• 5% Trimmed Mean: Recalculates mean after removing top and bottom 5% of scores to neutralize outlier distortion.
• Median: Positional 50th percentile; robust against skewness.
• Mode: Most frequent score; essential for discrete categorical choices.
Measures of Dispersion
• Variance ((s^2)): Squared dispersion metric.
• Interquartile Range ((IQR)): Middle 50% spread ((Q_3 - Q_1)).
• Standard Error of Mean (): Precision of sample mean as estimator of population parameter.
Higher-Order Distributional Moments: Skewness & Kurtosis
Evaluating the departure of an empirical distribution from the theoretical Gaussian bell curve requires analyzing third and fourth standardized moments:
| Distributional Metric | Mathematical Formulation & Baseline | Software Interpretation & Normality Benchmarks |
|---|---|---|
| 1. Skewness ((gamma_1)) |
Measures the degree of asymmetry about the mean:
|
• Positive Skew (> 0): Right tail is elongated; mean exceeds median (e.g., household income, corporate revenue). • Negative Skew (< 0): Left tail is elongated; median exceeds mean (e.g., age at retirement). • Rule of Thumb: Values between (-0.5) and (+0.5) indicate approximately symmetrical distribution. |
| 2. Kurtosis ((gamma_2)) |
Measures the peakedness and heavy-tailedness relative to normal curve:
text{Excess Kurtosis} = ≤ft[ (n(n+1)) / ((n-1)(n-2)(n-3)) ∑ ≤ft(frac{x_i - bar{x}}{s}right)^4 right
|
• Leptokurtic (> 0): Sharply peaked with fat, heavy tails; higher outlier probability (e.g., financial asset returns). • Platykurtic (< 0): Flatter peak with lighter, thin tails. • Mesokurtic (= 0): Standard normal bell curve. |
To statistically evaluate whether sample skewness or kurtosis departs significantly from normality, compute the test statistic:
Decision Rule: If or , reject the null hypothesis of normality at the 5% significance level ((p < 0.05)). For small samples ((n < 50)), use a stricter threshold of (|Z| > 2.58) ((p < 0.01)).
2. SOFTWARE PROCEDURES FOR DESCRIPTIVE SUMMARIES
Leading statistical suites provide dedicated analytical procedures tailored to exploratory data profiling:
SPSS Syntax Commands for Comprehensive Profiling
The following batch syntax executes an exhaustive descriptive audit on annual store revenues and customer footfall:
Microsoft Excel & R Computational Equivalents
| Analytical Task | Microsoft Excel Formula / Tool | R / tidyverse Script |
|---|---|---|
| Arithmetic Mean | =AVERAGE(A2:A100) |
mean(data$revenue, na.rm = TRUE) |
| Median | =MEDIAN(A2:A100) |
median(data$revenue, na.rm = TRUE) |
| Sample Standard Deviation | =STDEV.S(A2:A100) |
sd(data$revenue, na.rm = TRUE) |
| Interquartile Range (IQR) | =QUARTILE.INC(A2:A100, 3) - QUARTILE.INC(A2:A100, 1) |
IQR(data$revenue, na.rm = TRUE) |
| Skewness & Kurtosis | =SKEW(A2:A100) and =KURT(A2:A100) |
e1071::skewness(datarevenue) |
| Comprehensive Summary | Data Tab > Data Analysis > Descriptive Statistics | psych::describe(data$revenue) |
| Formal Normality Test | Requires add-in (e.g., Real Statistics) | shapiro.test(data$revenue) |
3. CROSS-TABULATION & CONTINGENCY TABLE ANALYSIS
When investigating simultaneous relationships between two categorical variables (nominal or ordinal), researchers employ Cross-Tabulation (also termed Contingency Tables or Two-Way Frequency Tables). Cross-tabulation organizes joint frequencies, enabling analysts to identify patterns of mutual association across demographic and operational segments.
Cell Percentage Configurations
• Column Percentages: Percentage of cases in a column falling into each row. Useful when the independent variable is in the columns.
• Total Percentages: Cell count divided by grand sample size (N).
Interpreting Relationships
Pearson's Chi-Square Test of Independence ((chi^2))
The Chi-Square test determines whether a systematic relationship exists between two categorical variables by contrasting observed cell frequencies ((O_{ij})) against the expected frequencies ((E_{ij})) that would occur under the null hypothesis of complete independence.
Methodological Assumptions of the Chi-Square Test
- Random Sampling: Data must be drawn randomly and independently from the population.
- Mutual Exclusivity: Each subject or transaction must fall into exactly one cell in the contingency table.
- Cochran's Minimum Expected Frequency Rule (1954):
- In an (r × c) table, at least 80% of the cells must have expected frequencies of 5 or more ((E_{ij} ≥ 5)).
- No cell may have an expected frequency of less than 1.0 ((E_{ij} ≥ 1.0)).
- Remedial Actions for Violations:
- For (2 × 2) tables with small expected counts ((E_{ij} < 5)), report Fisher's Exact Test instead of Pearson's Chi-Square.
- For larger tables, collapse adjacent conceptually related categories to increase cell frequencies.
Measures of Association for Categorical Data
While the Chi-Square test establishes whether an association is statistically significant, it is sensitive to sample size and does not quantify the strength of the relationship. Specialized association coefficients must be reported:
| Association Metric | Mathematical Formula | Operational Application & Interpretation Range |
|---|---|---|
| 1. Phi Coefficient ((phi)) | [ phi = √((chi^2) / (N)) | Exclusively utilized for tables. Bounded between 0.0 (no association) and 1.0 (perfect association). For larger tables, (phi) can mathematically exceed 1.0, rendering it uninterpretable. |
| 2. Cramér's (V) |
|
Universal standard for larger tables. Corrects for table dimensions; strictly bounded between 0.0 and 1.0. • (0.10 le V < 0.20): Weak association. • (0.20 le V < 0.40): Moderate association. • (V ge 0.40): Strong commercial relationship. |
| 3. Contingency Coefficient ((C)) |
|
Measures relationship strength, but its upper limit is strictly less than 1.0 (for a table, maximum ). |
| 4. Goodman & Kruskal's Lambda () |
|
Proportional Reduction in Error (PRE) measure. Quantifies how much prediction error of the dependent variable is reduced when the independent variable is known. Range: . |
4. DATA VISUALIZATION PRINCIPLES & GRAPHICAL ARCHITECTURES
Data visualization bridges raw statistical computation and executive comprehension. According to Edward Tufte's classical visualization doctrine, an effective statistical graphic must maximize the Data-Ink Ratio, avoid non-functional decoration ("chartjunk"), provide clear labeling, and preserve graphical integrity by avoiding 3D distortion or artificially truncated coordinate axes.
Categorical Visualizations
• Clustered Bar Charts: Displays two categorical variables side-by-side.
• Stacked / 100% Component Bar Charts: Illustrates compositional proportions across groups.
Continuous Distributional Charts
• Box-and-Whisker Plots: Depicts 5-number summaries () and outliers.
• Scatterplots: Explores bivariate co-variation and linear regression fit lines.
The Geometry of the Box-and-Whisker Plot
Specialized Commercial Visualizations
| Visualization Type | Structural Mechanism | Commercial & Quality Application |
|---|---|---|
| 1. Pareto Chart | Dual-axis chart combining sorted descending frequency bars with an ascending cumulative percentage line. | Root-cause quality control: Identifies the "vital few" defects (80%) versus the "trivial many" (20%) per Juran's Pareto Principle. |
| 2. Scatterplot with Lowess / OLS Fit | Plots paired coordinates ((X_i, Y_i)) overlaid with an Ordinary Least Squares linear regression line or Lowess smooth curve. | Evaluating pricing elasticity, advertising response functions, cost-volume-profit linear relationships. |
| 3. Correlation Heatmap | Color-coded square symmetric matrix where color intensity represents the magnitude and direction of Pearson's (r). | Screening financial portfolio assets for diversification; identifying multicollinearity among regression predictors. |
| 4. Ogive Curves | Plots cumulative frequencies (Less-Than and More-Than ogives) against class boundaries. | Graphical determination of median, quartiles, deciles, and wage distribution percentiles in labor economics. |
5. PUBLISHING, EXPORTING & REPORTING STATISTICAL OUTPUT
Statistical software generates extensive raw computational output that must be formatted into professional, publication-grade tables conforming to recognized academic standards (e.g., American Psychological Association - APA 7th Edition format) or corporate executive brief templates.
1. Table Number & Title: Bold table number (e.g., Table 1) followed by an italicized title on the subsequent line.
2. Horizontal Rule Protocol: Three horizontal lines only—one above the column headers, one below the column headers, and one at the bottom of the table. Vertical rules are strictly prohibited.
3. Decimal Consistency: Report means, standard deviations, and percentages to two decimal places; report correlation coefficients and (p)-values to three decimal places (e.g., (p = .014); report values below .001 as (p < .001)).
4. Table Notes: Add explanatory notes below the table defining abbreviations and reporting effect size metrics.
6. STEP-BY-STEP PRACTICAL INDUSTRY CASE ILLUSTRATIONS
A retail supermarket chain evaluated the monthly sales revenue (in lakhs of ₹) across 15 regional branch outlets:
Sample Revenue Data: 28, 32, 35, 36, 40, 42, 45, 46, 48, 50, 52, 55, 58, 62, 98
Required:
1. Calculate the Arithmetic Mean (), Median ((M)), and 5% Trimmed Mean.
2. Calculate the Sample Variance ((s^2)), Standard Deviation ((s)), and Standard Error of the Mean ().
3. Determine the First Quartile ((Q_1)), Third Quartile ((Q_3)), and Interquartile Range ((IQR)).
4. Given that sample skewness is (+1.72) with , compute the standardized skewness (Z)-score and evaluate whether the distribution departs significantly from normality.
5. Interpret the disparity between the Mean and the 5% Trimmed Mean.
Solution & Step-by-Step Working Notes:
Sorted sample ((n = 15)): 28, 32, 35, 36, 40, 42, 45, 46, 48, 50, 52, 55, 58, 62, 98.
Step 1: Measures of Central Tendency:
• .
• Mean: .
• Median: Position observation: .
• 5% Trimmed Mean: For (n = 15), trimming 5% from both tails removes the extreme single highest and lowest values (28 and 98), leaving (n = 13) observations with sum (727 - 28 - 98 = 601):
.
Step 2: Measures of Dispersion:
• Computing sum of squared deviations (∑ ^2):
| Store | Revenue ((X)) | ((X - 48.47)) | ((X - 48.47)^2) | Store | Revenue ((X)) | ((X - 48.47)) | ((X - 48.47)^2) |
|---|---|---|---|---|---|---|---|
| 1 | 28 | -20.47 | 419.02 | 9 | 48 | -0.47 | 0.22 |
| 2 | 32 | -16.47 | 271.26 | 10 | 50 | +1.53 | 2.34 |
| 3 | 35 | -13.47 | 181.44 | 11 | 52 | +3.53 | 12.46 |
| 4 | 36 | -12.47 | 155.50 | 12 | 55 | +6.53 | 42.64 |
| 5 | 40 | -8.47 | 71.74 | 13 | 58 | +9.53 | 90.82 |
| 6 | 42 | -6.47 | 41.86 | 14 | 62 | +13.53 | 183.06 |
| 7 | 45 | -3.47 | 12.04 | 15 | 98 | +49.53 | 2453.22 |
| 8 | 46 | -2.47 | 6.10 | Total | 727 | 0.00 | 3861.70 |
• Sample Variance: (s^2 = (∑ ^2) / (n - 1) = (3861.70) / (14) = mathbf{275.84}).
• Sample Standard Deviation: .
• Standard Error of Mean: .
Step 3: Quartiles and IQR:
• (Q_1) position = (0.25(n + 1) = 4^{th}) item: .
• (Q_3) position = (0.75(n + 1) = 12^{th}) item: .
• .
Step 4: Skewness Normality Test:
Step 5: Managerial Interpretation:
The standard mean (₹ 48.47 lakhs) is heavily inflated by the flagship store outlier (₹ 98 lakhs). The 5% Trimmed Mean (₹ 46.23 lakhs) aligns closely with the Median (₹ 46.00 lakhs), demonstrating that the Trimmed Mean provides a vastly superior, robust reflection of typical branch earning capacity.
A digital payments fintech research group conducted a nationwide customer survey of 300 retail shoppers to examine whether Preferred Payment Mode is independent of Age Demographic Group.
Observed Cross-Tabulation Data ((O_{ij})):
• Young Adults (Under 30): UPI = 70, Card = 20, Cash = 10 (Total = 100).
• Middle-Aged (30 to 50): UPI = 45, Card = 40, Cash = 15 (Total = 100).
• Seniors (Above 50): UPI = 15, Card = 30, Cash = 55 (Total = 100).
Required:
1. Construct the complete contingency table displaying observed counts and marginal totals.
2. Compute the matrix of expected frequencies under the null hypothesis of independence.
3. Compute Pearson's Chi-Square Test Statistic ((chi^2)) and determine degrees of freedom.
4. At a 5% level of significance ( for (df = 4)), state the formal hypothesis decision.
5. Calculate Cramér's (V) and interpret the strength of association.
Solution & Step-by-Step Working Notes:
Step 1: Observed Frequencies ((O_{ij})) & Marginal Totals:
Row 1 (Young): (O_{11} = 70, O_{12} = 20, O_{13} = 10 implies R_1 = 100).
Row 2 (Middle): (O_{21} = 45, O_{22} = 40, O_{23} = 15 implies R_2 = 100).
Row 3 (Senior): (O_{31} = 15, O_{32} = 30, O_{33} = 55 implies R_3 = 100).
Column Totals: (C_1 = 70 + 45 + 15 = 130); (C_2 = 20 + 40 + 30 = 90); (C_3 = 10 + 15 + 55 = 80).
Grand Total (N = 300).
Step 2: Calculate Expected Frequencies ():
Since all row totals (R_i = 100) and (N = 300), :
• For UPI (Col 1): .
• For Card (Col 2): .
• For Cash (Col 3): .
Cochran's Rule Check: All expected frequencies exceed 26.0 (well above 5.0). Assumption satisfied!
| Cell ((i, j)) | Observed ((O)) | Expected ((E)) | ((O - E)) | ((O - E)^2) | (((O - E)^2) / (E)) |
|---|---|---|---|---|---|
| Young - UPI | 70 | 43.333 | +26.667 | 711.129 | 16.411 |
| Young - Card | 20 | 30.000 | -10.000 | 100.000 | 3.333 |
| Young - Cash | 10 | 26.667 | -16.667 | 277.789 | 10.417 |
| Middle - UPI | 45 | 43.333 | +1.667 | 2.779 | 0.064 |
| Middle - Card | 40 | 30.000 | +10.000 | 100.000 | 3.333 |
| Middle - Cash | 15 | 26.667 | -11.667 | 136.119 | 5.104 |
| Senior - UPI | 15 | 43.333 | -28.333 | 802.759 | 18.525 |
| Senior - Card | 30 | 30.000 | 0.000 | 0.000 | 0.000 |
| Senior - Cash | 55 | 26.667 | +28.333 | 802.759 | 30.103 |
| Total | 300 | 300.000 | 0.000 | - | (chi^2 = 87.290) |
Step 3: Degrees of Freedom:
Step 4: Hypothesis Testing Decision:
The calculated Chi-Square statistic is (chi^2 = 87.29).
The critical value from Chi-Square distribution tables at with (df = 4) is (chi^2_{0.05, 4} = 9.488).
Since ((p < 0.001)), we decisively reject the Null Hypothesis ((H_0)) of independence. Preferred payment mode is significantly dependent on customer age demographic.
Step 5: Strength of Association (Cramér's (V)):
7. COMPREHENSIVE SELF-ASSESSMENT MODULE
Select the most appropriate option. Complete solution key with technical rationale follows.
1. In SPSS Statistics, which exploratory procedure provides formal Kolmogorov-Smirnov and Shapiro-Wilk tests of normality?
(a) Frequencies (b) Descriptives (c) Explore (d) Means
2. In a statistical boxplot, the upper and lower boundaries of the central rectangular box correspond exactly to:
(a) Mean and Standard Deviation (b) First Quartile ((Q_1)) and Third Quartile ((Q_3))
(c) Minimum and Maximum values (d) 95% Confidence Intervals
3. What is the degrees of freedom for a Chi-Square test of independence conducted on a contingency table?
(a) 12 (b) 7 (c) 6 (d) 5
5. Which measure of association is mathematically appropriate for evaluating the strength of a relationship in an arbitrary table?
(a) Phi Coefficient (b) Cramér's (V) (c) Spearman's Rho (d) Durbin-Watson
6. If a distribution of customer waiting times has a sample skewness of (+0.85) with a standard error of (0.20), the distribution is:
(a) Perfectly symmetrical (b) Statistically significantly positively skewed ((p < 0.05))
(c) Platykurtic (d) Negatively skewed
7. The specialized quality control chart that combines a descending bar graph of defect causes with an ascending cumulative percentage line is called a:
(a) Run Chart (b) Gantt Chart (c) Pareto Chart (d) Scatter Plot
8. In APA 7th edition formatting rules for statistical tables, which type of borders are strictly prohibited?
(a) Top horizontal rules (b) Vertical gridlines (c) Bottom table lines (d) Header dividers
9. If a contingency table has cell expected frequencies below 5, which exact test should be interpreted instead of Pearson's Chi-Square?
(a) Mann-Whitney U Test (b) Fisher's Exact Test (c) Wilcoxon Test (d) Kruskal-Wallis
10. In R, which plotting package implements the "Grammar of Graphics" framework separating data, aesthetics, and geometric layers?
(a) lattice (b) base graphics (c) ggplot2 (d) plotly
1. (c) Explore: The
EXAMINE / Explore procedure generates the formal normality tests.2. (b) (Q_1) and (Q_3): The box defines the interquartile range from the 25th to 75th percentiles.
3. (c) 6: (df = (4 - 1)(3 - 1) = 3 × 2 = 6).
4. (c) 20%: At least 80% of cells must have , meaning no more than 20% can fall below 5.
5. (b) Cramér's (V): Adjusts for degrees of freedom and is bounded between 0 and 1 for tables of any size.
6. (b): , confirming significant positive skewness.
7. (c) Pareto Chart: Standard 80/20 root cause analysis visualization.
8. (b) Vertical gridlines: APA style strictly bans vertical lines.
9. (b) Fisher's Exact Test: Calculates exact hyper-geometric probabilities for sparse contingency tables.
10. (c) ggplot2: Hadley Wickham's implementation of Leland Wilkinson's grammar of graphics.
- Explain the mathematical rationale and operational utility of the 5% Trimmed Mean. Under what specific data conditions does the Trimmed Mean provide superior managerial insight compared to the standard Arithmetic Mean and Median?
- State the complete theoretical assumptions of the Chi-Square Test of Independence. What methodological options are available to a data analyst when the minimum expected cell count assumption is violated?
- Contrast the Phi Coefficient, Cramér's (V), and Contingency Coefficient. Why is Cramér's (V) the universally preferred metric of association for contingency tables larger than ?
- Detail the structural geometry of a Tukey Boxplot. How does a boxplot visually represent central tendency, dispersion, skewness, and extreme outliers simultaneously?
- Describe the core principles of Edward Tufte's data visualization doctrine. Provide examples of common "chartjunk" and graphical distortions frequently encountered in commercial corporate dashboards.
Case Scenario 1: Retail Store Customer Satisfaction & Promotion Analysis
A department store chain surveys 250 shoppers to evaluate whether customer satisfaction level is associated with participation in the store's festive promotional loyalty program.
• Loyalty Members (150 shoppers): Satisfied = 105, Neutral = 30, Dissatisfied = 15.
• Non-Members (100 shoppers): Satisfied = 35, Neutral = 35, Dissatisfied = 30.
Required:
1. Construct the contingency table with observed counts and marginal totals.
2. Compute row percentages and column percentages for each cell.
3. Compute expected cell frequencies under the null hypothesis of independence.
4. Calculate the Chi-Square test statistic ((chi^2)), specify degrees of freedom, and evaluate significance at (Critical value for (df = 2) at 1% level is 9.210).
5. Compute Cramér's (V) and compose an executive interpretation advising the retail marketing director.
Case Scenario 2: Exploratory Salary Distribution Audit Across Executive Tiers
A management consulting firm compiled monthly compensation packages (in thousands of ₹) across three tiers of corporate staff:
• Junior Associates ((n = 8)): 35, 38, 40, 42, 45, 48, 50, 52.
• Mid-Level Managers ((n = 8)): 65, 70, 72, 75, 78, 80, 85, 120.
• Senior Executives ((n = 8)): 140, 150, 155, 160, 165, 175, 180, 310.
Required:
1. For each group, compute: Mean, Median, (Q_1, Q_3), and (IQR).
2. Compute the Upper Inner and Outer Fences for each group using Tukey's criteria.
3. Identify any mild or extreme outliers in each managerial tier.
4. Sketch the comparative side-by-side boxplots illustrating the compensation disparities and outlier points.
5. Formulate an APA-style summary table presenting the complete descriptive statistics for a compensation committee presentation.
Finished reading Module 2?
Save this module as a PDF for offline study or advance to the next module.