Подтвердите e-mail

Для публикаций, комментариев, реакций и сообщений подтвердите адрес.

←

Публикация

Предыдущая публикация
Hester Duffy

It's not the rows, it's the columns. There should be pre and post scores for both ai and lecture for each participant. There aren't. They only report one session per participant.

111
Elixir of Progress

But there are pre and post scores for each lecture for AI and not AI. Using visidata, I can see distinct IDs repeated with the "lect_AI" labeling field. So, is your objection that you'd prefer a wide data format rather than a long data format?

Обсуждение

3 прямых ответов · 16 сообщений

Hester Duffy

Ah, wait, I see what you mean! They've... Recorded their participants twice? Thus removing the ability to do a within participants comparison, despite that being part of their design?

Ответ для Hester Duffy
Hester Duffy

Ok, so I was wrong about them losing half of their data; it's just been presented and analysed in a really weird way. Still not good research practice, but less bad than I thought.

Ответ для Hester Duffy
Elixir of Progress

So long as we set allegations of p-hacking aside, I agree this paper isn't the best I've ever read. It's just an effect strong enough to be worth investigating.

Ответ для Elixir of Progress
Hester Duffy

I would like to see it investigated better; a better comparison (workshop rather than lecture, for example), and a much better analysis. I'm very wary of regression analyses which don't have very clear justifications; it's fairly easy to bung in a lot of variables and get something out.

Ответ для Hester Duffy
Elixir of Progress

Yeah, that's how I feel too (although I think I'm less afraid of regressions because of the methodologies I prefer to use).

Ответ для Hester Duffy
Elixir of Progress

To the first part yes: each id should appear twice. To the second part no, its not any more difficult.

Ответ для Elixir of Progress
Hester Duffy

But they haven't done within participants comparisons; they've compared mean post scores for each group to the total pre scores, instead of just doing a paired comparison, which would be stronger and make more sense. As you said, they've treated it as independent samples.

Ответ для Hester Duffy
Elixir of Progress

I'm back to a condition where I can't consider methodology (I'm on a bus heading to a glacier) but my reply was to your comment about how laying out the data this way prevents all in cohort comparison. It is still trivial, you just pivot on the label. If that's not what you went, sorry.

Ответ для Elixir of Progress
Hester Duffy

I'm away from my work laptop too; I might have a play at some point and see what happens if you just do a sensible repeated measures ANOVA on it (although I don't think they've recorded order in there, which is a shame, I would have liked to check for order effects too).

Ответ для Hester Duffy
Elixir of Progress

You can probably recover order effects. I'm all for extra numeric replication. I'm not a fan of frequentist approaches and because I'm not a professor, I don't have to be :)

Ответ для Elixir of Progress
Hester Duffy

Yeah, I think they're basically implied by the order of the entries, which seems like very poor data management to me, but should be usable.

Ответ для Elixir of Progress
Hester Duffy

(Also, glacier? Very cool, literally but also figuratively!)

Hester Duffy

No; there is one pre and one post score for each student, and a variable defining whether they did AI or lecture. That's one session each. But they were supposed to do two sessions each. They should have two pre and two post scores each (one AI, one lecture), not one.

Elixir of Progress

I think the long-form-fixed-shape format is extremely common now because of the benefits of a uniform shape when using it with dataframe libraries and many R and Julia libraries expect this sort of shape.

Ответ для Elixir of Progress
Hester Duffy

I'm an SPSS girl myself, and one of the first lessons we drill into students is "one row per participant and one participant per row". Do R and Julia not cope with missing data?

Ответ для Hester Duffy
Elixir of Progress

They cope just fine. It's just a data convention. There's really no right or wrong, and I think gradually the downstream effects of Kruschke and McElreath and others are making it increasingly common.