2022-07-15 Michael Nielsen PNAS CI paper comments

Michael is in San Francisco (@michael_nielsen): Fascinating claims in this article on collective intelligence (via @anderssandberg): https://www.pnas.org/doi/10.1073/pnas.2005737118 https://twitter.com/michael_nielsen/status/1547969157752246278/photo/1

The basic idea is to define a common factor across multiple tasks, as with g for individuals: https://twitter.com/michael_nielsen/status/1547969877398343680/photo/1

IIRC working memory and g have a very high correlation, suggestive that working memory capacity may be the underlying mechanism. Bit surprised that's not mentioned here.

(I am certainly not an expert on g, though, maybe my info is out of date or mistaken!) https://twitter.com/michael_nielsen/status/1547970724635086849/photo/1

I so enjoy the back-and-forth of ideas:

(Ernst Mayr's wonderful phrase "one long argument" applies across science) https://twitter.com/michael_nielsen/status/1547971213372186626/photo/1

Meta-analyses always make me twitchy (they drown out signal, & are often used in a kind of magic fairy dust way). But probably necessary here. https://twitter.com/michael_nielsen/status/1547971762268225536/photo/1

Something I wonder - and doesn't appear to be addressed here - is how stable CI is, and how subject to change due to interventions.

I think everyone's been in groups which slightly changed the way they worked, and all of a sudden boom things are far better or far worse.

Hmm, POGS. From the Department of Unfortunate Acronyms. https://twitter.com/michael_nielsen/status/1547972514759843840/photo/1

Fun - hold-out a task, use CI to predict performance on that task. Works surprisingly well!

(Though calling this "strong evidence" is very amusing. I might go with "suggestive", though I'd worry I was exaggerating...) https://twitter.com/michael_nielsen/status/1547973788658122756/photo/1

This kind of thing just makes me grind my teeth. You haven't "established" any such thing.

I think I'll eschew further comment on quality of evidence (you get the point). https://twitter.com/michael_nielsen/status/1547974188983431169/photo/1

Fascinating. A negative effect of age diversity! I'll bet that often reverses under better-designed circumstances (stable groups, more serious problems, more long-range work). https://twitter.com/michael_nielsen/status/1547974582367268865/photo/1

Anyway it's a fun paper, posing many interesting questions! I'm pretty skeptical of anything like a group CI analogous to g - I suspect simple process adjustments ("culture") will cause wild swings in CI. But it's an interesting line of work!

(Actually, it'd be fun and I suspect rather easy to demonstrate that interventions can cause sizeable swings in CI.)