A basic primer to hypothesis testing (part 1)

Summarizing an earlier post, I'd say that the basis of scientific enquiry is to model systems (a part of the world around us) by making falsifiable empirical claims (hypotheses) about it. These hypotheses are then formally validated by first constructing a well defined experiment around them that generates data from the system under observation, and then statistical analysis of said data. Post analysis, these hypotheses are either rejected or provisionally supported, along with revision in our models if necessary.

In this post, I would like to chalk out the formal validation process of the hypothesis with the example of a fear conditioning experiment. [TODO: add link]

Suppose you run a fear conditioning experiment and collect pupillometry data along with it. A data processing pipeline (perhaps developed by someone like me) will take in the eye-tracking data generated for the experiment and after multiple stages of preprocessing and modelling generate output pupil size responses for each of the conditions CS+ (conditioned stimulus followed by shock, here rotated: square), CS- (conditioned stimulus not followed by shock, here: square)

Consider that the final output looks something like this:

participant_id	cs_p_response	cs_m_response
p_001	0.838	0.256
p_002	0.842	0.278
p_003	0.657	0.392
p_004	0.769	0.138
p_005	0.800	0.169

Our primary aim in this exercise is to prove that post-conditioning, the participant does associate CS+ with the aversive stimulus, which would be indicated through increased CS+ response in comparison to the CS- response. So, we construct two hypotheses:

H₀: There is no effect or no difference in CS- and CS+ responses.
H₁: There is an positive effect or a positive difference in CS- and CS+ responses.

**One important point to note about hypothesis testing is that we always test against the null hypothesis instead of "trying to prove the alternate hypothesis."

Now, before we can statistically prove/disprove the null hypothesis, it should do us good to just try and plot our data to eyeball if there is some difference in the distributions of CS+ and CS- responses.

Looking at the plot, one can clearly see that the CS+ responses are higher in general (the blue curve on the right) then the CS- responses (the orange curve on the left). This supports our alternate hypothesis H₁. 

---

Our job would've ended here if this was to be considered enough proof of validity of H₁ (rather, invalidity of H₀). Alas, that is not the case and visual inspection is nowhere rigorous enough to conclude our study. We must employ actual statistical tests (like the Student's t-test) to prove our hypothesis. 

We need to quantify the difference the two distributions corresponding to the two groups in order to show that the difference is not (distributed) how we expect it to be (difference = 0, or rather centered around 0).

[TODO: expand]