November 2001
- Digimask Technology. Matt Owen
- Digimask Technology. Ian Macintosh
- Call for submissions to Game Programming Gems 3 Andrew Kirmse
- Virtual environments for education? Edward Glowacki
- Virtual environments for education? Hans-Henrik Staerfeldt
- Virtual environments for education? Bruce Mitchener
- Virtual environments for education? Ola Fosheim Grøstad
- Virtual environments for education? Dr. Cat
- Virtual environments for education? Mats Lidstrom
- Playing NPCs Daniel.Harman@barclayscapital.com
- A Hello, and thanks... o. rchaeus
- NPCS Mr Dylan Tovey
- Story in MM*s [was ] Joe Andrieu
- player-driven content? Sasha Hart
- player-driven content? Paul Schwanz
- player-driven content? Bruce Mitchener
- (no subject) J C Lawrence
- (no subject) Koster, Raph
- (no subject) Peter Tyson
- WebRPG Travis Casey
- [TECH][SURVEY] non-mainstream languages? Bruce Mitchener
- [TECH][SURVEY] non-mainstream languages? Brian Hook
- [TECH][SURVEY] non-mainstream languages? Nathan F. Yospe
- [TECH][SURVEY] non-mainstream languages? Mats Lidstrom
- [TECH][SURVEY] non-mainstream languages? Bruce Mitchener
- "sweeping change"? Sellers, Mike
- "sweeping change"? Michael Tresca
- "sweeping change"? Sellers, Mike
- "sweeping change"? Michael Tresca
- "sweeping change"? Freeman, Jeff
- "sweeping change"? Matt Mihaly
- "sweeping change"? Sellers, Mike
- Re[4]: Expectations of in-game reality Travis Casey
- Cynical at a young age T.A.J.BARTON
- Tabletop RPGs and Inspiration (was: no subject) Brian 'Psychochild' Green
- Tabletop RPGs and Inspiration (was: no subject) John Mariotti
- Tabletop RPGs and Inspiration (was: no subject) Matt Chatterley
- Tabletop RPGs and Inspiration (was: no subject) Sean K
- Tabletop RPGs and Inspiration (was: no subject) Don Healey
- Some survey results... Andrew Wilson
- Some survey results... Matt Mihaly
- Some survey results... Jeremy Noetzelman
- Some survey results... Andrew Wilson
- Some survey results... Dr. Cat
- Some survey results... Andrew Wilson
- Some survey results... Dr. Cat
- Some survey results... Derek Snider
- Some survey results... Andrew Wilson
- Some survey results... Matt Mihaly
- Role-Playing Games Are Not Dead Michael Tresca
- Role-Playing Games Are Not Dead Dave Rickey
- Role-Playing Games Are Not Dead Ryan S. Dancey
- Role-Playing Games Are Not Dead Richard Aihoshi aka Jonric
- Role-Playing Games Are Not Dead Leland Hulbert II
- Role-Playing Games Are Not Dead Richard Aihoshi aka Jonric
- Role-Playing Games Are Not Dead Matt Mihaly
- Role-Playing Games Are Not Dead Paul Schwanz
- Role-Playing Games Are Not Dead Dave Rickey
- Role-Playing Games Are Not Dead Sellers, Mike
- Role-Playing Games Are Not Dead Ryan S. Dancey
- Role-Playing Games Are Not Dead Valerio Santinelli
- Role-Playing Games Are Not Dead Koster, Raph
- Role-Playing Games Are Not Dead Michael Tresca
- Role-Playing Games Are Not Dead Brian Hook
- Role-Playing Games Are Not Dead Lars Duening
- Role-Playing Games Are Not Dead Ola Fosheim Grøstad
- Role-Playing Games Are Not Dead Phillip Lenhardt
- Role-Playing Games Are Not Dead Lars Duening
- Role-Playing Games Are Not Dead Matt Mihaly
- Role-Playing Games Are Not Dead Travis Casey
- Role-Playing Games Are Not Dead Matt Mihaly
- Role-Playing Games Are Not Dead Travis Casey
- Role-Playing Games Are Not Dead Don Healey
- Role-Playing Games Are Not Dead Michael Tresca
- Role-Playing Games Are Not Dead Matt Mihaly
- Role-Playing Games Are Not Dead Paul Schwanz
- Role-Playing Games Are Not Dead Travis Casey
- Role-Playing Games Are Not Dead Phillip Lenhardt
- Role-Playing Games Are Not Dead Don Healey
- Role-Playing Games Are Not Dead Koster, Raph
- Role-Playing Games Are Not Dead Ling Lo
This was posted a while back:
http://groups.google.com/groups?selm=dwendt-0603001601410001%40statmac63.mps.ohio-state.edu&output=gplain
---<cut>---
From: dwendt@consultant.com (David Wendt)
Subject: A Response To The Adventure Game Industry Markey Research Summary
Newsgroups: rec.games.frp.industry
Date: 2000/03/06
From: David Wendt, Ph.D.
Professor of Statistics
Statistical Consultant
Permissions: This file is based on data presented in the Adventure
Game Industry Market Research Summary. The file in question is
Copyright 2000, Wizards of the Coast. The file in question had
permission granted to be freely redistributed in or quoted in whole
or in part, provided that this attribution remained intact.
With regard to Mr. Dancey's Market Research Summary:
While I applaud Wizards of the Coast, and Mr. Dancey in particular,
for their openness with their market data, I feel that must express
some concerns with the manner in which they presented their
information. While Market Research methodologies are commonly
accepted for internal corporate use, one must be careful when
presenting them in a public forum. In this document, I plan to
address my concerns with this particular summary one at a time. I
encourage my colleagues to comment on my discussion and in
particular on a number of open issues I point out in the text.
With regard to the 'Pokemon Effect': While I respect the decision
not present information on the trading card game (henceforth
referred to as TCG) component of the data, your survey was intended
to "determine information about [TCGs], [RPGs] and [MWGs]". This
sends up an immediate flag with respect to my training. Any survey
or test is set up to gather specific information. When one changes
the use of the data, one is in danger of invalidating the
information. Thus, we must already consider the rest of your
conclusions with a grain of salt.
My second major concern is related to your decision to exclude
population based on age. Your acknowledgement of the impact of this
exclusion was a good first step. However, a great many of your
conclusions are age- and time- based. As a result, you have
potentially introduced a great deal of bias to your data. This is a
second reason to be wary of the conclusions presented in your
document.
Additionally, you claim that the survey "accurately represents the
US population as a whole". I will assume this is a misstatement.
The US population includes a large population in the over 35 group.
You might argue that your survey accurately represents the 12 to 35
age group in the United States. This is also possibly incorrect,
however, as the screener and follow-up survey may have produced
further bias based on the information provided to the respondents.
Were the surveys identified as being sponsored by WOTC or some
related organization? In addition, non-compliance also tends to
bias such surveys (who is the sort of person who would take the time
to fill them out?). This non-compliance bias is well documented and
is part of the recent Census versus Sampling debates, for those who
wish to read more on this issue.
I will address further concerns related to the relationship between
age and time further later in this document.
Let us consider your conclusions in Section3: Basic Demographics.
You either have been careless in your explanation of your
conclusions, or have omitted some of the results you base those
conclusions on. While you provide single level crossover values,
you do not provide the probability of multiple crossovers (defined
as, for instance, a TRPG gamer plays two or more of the other
categories.) Comparing the crossover values you did provide to the
percent of women in gaming (which you call significant), the values
seem comparable. This would imply that by your definitions, these
crossover percentages are significant. (As an aside, 'significant'
has a very specific statistical meaning, so one should be careful
when using it in a public document.)
Otherwise, I am a bit concerned with the conclusion you draw. I am
not at all convinced that the fact there is little crossover means
that extending a brand can be successful. I would imagine there are
psychological factors to be accounted for, and would welcome the
comments of anyone trained in such matters on this issue. Further,
I suspect there may be gender buying trends that you have not
accounted for when you say that "females, as a group, spend less
than males". Again, I choose not to comment further, but encourage
those with appropriate training to do so.
Finally, for this section, while your conclusion that gaming is an
adult hobby is well supported by your data, I remind the reader
again that a significant portion of the populace has not been
included. I suspect that the percentages of gamers in the lower age
brackets has been unduly inflated as a result. Further, your final
sentence in this section, "...the existing group of players is aging
and not being refreshed by younger players at the same rate as in
previous years" is completely unfounded. First, without previous
years market data, we have no benchmark to compare these number to.
Secondly, later in your own document, you observe that gamers join
the hobby at various stages of life. This will directly increase
percentages as age increases by definition.
In sections 4 and 5, I again encourage the reader to be aware that
all of these percentages as biased by the exclusion of respondents
over 35. In particular, levels of education and marital status have
direct relationships to age. A 12 year old is much less likely to
have done post-graduate work or to be partnered than a 35 year old.
This must be considered when reviewing the results.
I have additional concerns about the results presented in section 5.
First, I must revisit my concerns about potential bias due to the
attributions of the survey. If response were biased by perceived
ownership (that is if the respondent believed he was responding a
specific company), then all the percentages are in doubt, but in
particular those about "past game experiences". Different game
companies have historically had different attitudes about a number
of the experiences the respondents were queried on. This should be
kept in mind when considering the results.
Secondly, the amount of spending is likely correlated to age. Due
to the impact of education and employment, older gamers are likely
to have more expendable income. Thus increasing their total
spending. Further, the longer a particular gamer has been in the
hobby, the older they are, thus taking us back to the earlier
argument. As a result, subdividing spending by age is of minimal
information, and likewise with the D&D specific spending.
Regarding your monthly spending comments, I will simply encourage
you to look to hypothesis testing. I suspect that the perceived
differences are not significant. That is, due to variations in the
data, there is not sufficient evidence to say that there are
spending differences. Such tests are well documented in basic
statistics texts, and as a result, I will comment no further on
them.
I do not find the results on the spending of current versus lapsed
players overly interesting. I suggest an alternate measure to you.
That is the Mean Total non-D&D Spending per non-D&D game played.
Based on your numbers, Mean Total non-D&D spending is ($1273 - $895
=) $378 for current players and $1068 for lapsed players. Dividing
these by the Number of non-D&D games played (1.2 and 2.3
respectively), gives the following:
Mean Total non-D&D Spending
per non-D&D RPG Played: $315 / $464
These numbers are much closer indicating that perhaps there is
little to no difference in non-D&D playing. It also suggests that
perhaps RPGs other than D&D represent less of an investment for
players. I also suggest that there may be a bias inflicted by the
age truncation that may explain the difference between current and
lapses spending. Further, lapsed players may be overestimating
their spending and current players underestimating theirs. This is
a psychological factor that I would encourage professional or
academic comment on.
I am a little concerned about your percentages regarding "what games
TRPG players play" and "retail type". First, does the 35 or older
truncation bias these results? Do any of the other potential
sources of bias discussed in this document affect these numbers?
These are points for consideration. Relating more specifically to
what games are being played, the choice of games over companies is
disturbing and introduces a newer source for error. I suggest that
for future surveys, this sort of question be replaced by "Do you
play TRPGs from the following companies? Wizards of the Coast,
White Wolf, ..., Other, Unknown." A number of solid games were
excluded from your list and not all companies were included equally.
As a result, this data, as it stands, is useful for little more than
trivia (ignoring any bias issues). Percentages for games from the
same company cannot be added with any meaning, due to overlap, nor
can this be used for any measure of marketshare.
I am curious about how you moved from "on a scale from 1 to 5" to
percentages in the question where players where asked to describe
aspects of their games. While the data points are potentially of
great interest with respect to game design, without more information
on how the data was gathered and the results computed, I believe
this section tells us very little.
In conclusion, I again applaud Wizards of the Coast for their
openness with their market data. I encourage other members of the
gaming community to be equally forthcoming. However, due to
decisions made in the data collection phase and in the results
development phase, I caution anyone who reads this survey to be
cautious in drawing any conclusions. There is not only a danger of
significant bias, but important correlations in the data have been
seemingly ignored. Market research is a powerful internal corporate
tool, but must be taken as such when released to the public.
David Wendt, Ph.D.
---<cut>---
--
| Ling Lo
_O_O_ ling@slimy.com - Role-Playing Games Are Not Dead Ryan S. Dancey
- Role-Playing Games Are Not Dead Paul Schwanz
- Role-Playing Games Are Not Dead Michael Tresca
- [DGN] The Human Condition Kwon Ekstrom
- Multi-protagonist stories. Paul Schwanz
- Multi-protagonist stories. Matt Mihaly
- licensing Adam Martin
- Tabletop RPGs and Inspiration (was: no subject) Sami Kosonen
- Licensing PnP RPGs was "sweeping change"? Brian Hook
- Licensing PnP RPGs was "sweeping change"? Richard Aihoshi aka Jonric
- Licensing PnP RPGs was "sweeping change"? Freeman, Jeff
- Good Writing (was: Role-Playing Games Are Not Dead) Brian 'Psychochild' Green
- New laws. (was: Player Manipulation of Environment) Marian Griffith
- New laws. (was: Player Manipulation of Environment) Hans-Henrik Staerfeldt
- New laws. (was: Player Manipulation of Environment) John Buehler
- New laws. (was: Player Manipulation of Environment) Bruce Mitchener
- New laws. (was: Player Manipulation of Environment) Marian Griffith
- New laws. (was: Player Manipulation of Environment) Hans-Henrik Staerfeldt
- New laws. (was: Player Manipulation of Environment) Bruce Mitchener
- New laws. (was: Player Manipulation of Environment) Marian Griffith
- New laws. (was: Player Manipulation of Environment) Mark Eaton
- New laws. (was: Player Manipulation of Environment) Marian Griffith
- New laws. (was: Player Manipulation of Environment) Hans-Henrik Staerfeldt
- New laws. (was: Player Manipulation of Environment) Travis Casey
- New laws. (was: Player Manipulation of Environment) Hans-Henrik Staerfeldt
- New laws. (was: Player Manipulation of Environment) Travis Casey
- New laws. (was: Player Manipulation of Environment) Andrew Hefford
- New laws. (was: Player Manipulation of Environment) Paul Schwanz
- New laws. (was: Player Manipulation of Environment) Matt Mihaly
- New laws. (was: Player Manipulation of Environment) Marian Griffith
- New laws. (was: Player Manipulation of Environment) Paul Schwanz
- Storied Games Paul Schwanz
- Storied Games amanda@alfar.com
- Storied Games Lee Sheldon
- Storied Games Dave Rickey
- Storied Games Travis Casey
- Storied Games Paul Schwanz
- Storied Games Marian Griffith
- Storied Games Daniel.Harman@barclayscapital.com
- Storied Games Lee Sheldon
- Storied Games John W. Pierce
- Storied Games Lee Sheldon
- Storied Games jsmithn@hotmail.com
- Storied Games Matt Mihaly
- Storied Games Miroslav Silovic
- Storied Games Derek Licciardi
- Storied Games Matt Mihaly
- Storied Games amanda@alfar.com
- Storied Games Matt Mihaly
- Storied Games amanda@alfar.com
- Storied Games Matt Mihaly
- Player Manipulation of Environment/New Laws Paul Schwanz
- Player Manipulation of Environment/New Laws Bruce Mitchener
- Player Manipulation of Environment/New Laws Paul Schwanz
- Player Manipulation of Environment/New Laws Marian Griffith
- Player Manipulation of Environment Paul Schwanz
- Player Manipulation of Environment Paul Schwanz
- Player Manipulation of Environment Matt Mihaly
- Player Manipulation of Environment Andrew Hefford {Coregen}
- Player Manipulation of Environment Eli Stevens
- Player Manipulation of Environment Jasper McChesney
- Player Manipulation of Environment Paul Schwanz
- Player Manipulation of Environment Adam Martin
- Player Manipulation of Environment Jasper McChesney
- Player Manipulation of Environment Hans-Henrik Staerfeldt
- Player Manipulation of Environment Ling Lo
- Player Manipulation of Environment rayzam
- Player Manipulation of Environment Hans-Henrik Staerfeldt
- Player Manipulation of Environment Marc Hernandez
- Fw: AVATARS2001 this weekend Frank Crowell
- [NEWS] Lineage: The Blood Pledge Interview Dave Kennerly
- [NEWS] Lineage: The Blood Pledge Interview Dave Kennerly
- Storytelling and Professionals (was: ) Kathleen Foley
- Storytelling and Professionals (was: ) Lee Sheldon
- Storytelling and Professionals (was: ) Dave Rickey
- Storytelling and Professionals (was: ) Lee Sheldon
- not about telling stories Joe Andrieu
- not about telling stories Freeman, Jeff
- not about telling stories Travis Casey
- Storytelling and Professionals (was) Lee Sheldon
- A Non-Cumulative Character MMORPG? Heresy! Paul Schwanz
- Storytelling and Professionals (was) Ananda Dawnsinger
- Asheron's Call, Story and Population Density. Zak Jarvis
- Asheron's Call, Story and Population Density. Sasha Hart
- Asheron's Call, Story and Population Density. Christopher Kohnert
- Asheron's Call, Story and Population Density. Sasha Hart
- Asheron's Call, Story and Population Density. Christopher Kohnert
- Asheron's Call, Story and Population Density. Lee Sheldon
- Tech: Pathfinding with Rooms Sanxion
- Tech: Pathfinding with Rooms vognsen@post10.tele.dk
- Tech: Pathfinding with Rooms David Bennett
- Tech: Pathfinding with Rooms Sanxion
- Tech: Pathfinding with Rooms Christopher Kohnert
- Tech: Pathfinding with Rooms Per Vognsen
- Tech: Pathfinding with Rooms David Bennett
- Tech: Pathfinding with Rooms Eli Stevens
- Tech: Pathfinding with Rooms Christopher Kohnert
- Tech: Pathfinding with Rooms Nicholas E. Walker
- Tech: Pathfinding with Rooms Marcus Johansson
- Tech: Pathfinding with Rooms Hans-Henrik Staerfeldt
- Tech: Pathfinding with Rooms Leland Hulbert II
- [NEWS] Lineage: The Blood Pledge Interview - Richard's blunder :) Mathieu Castelli
- Subject: New laws. (was: Player Manipulation of Environment) John Robert Arras
- Meridian 59 purchased by developers Andrew Kirmse
- Meridian 59 purchased by developers James C. Nugen
- Meridian 59 purchased by developers Frank Crowell
- [STORY] Story and population size Adam Martin
- [STORY] Story and population size Derek Licciardi
- [STORY] Story and population size Christopher Kohnert
- [STORY] Story and population size Timothy Dang
- [STORY] Story and population size Derek Licciardi
- [STORY] Story and population size Vincent Archer
- [STORY] Story and population size Christopher Kohnert
- [STORY] Story and population size John Buehler
- [STORY] Story and population size Derek Licciardi
- [STORY] Story and population size John Buehler
- [STORY] Story and population size Jeff Freeman
- [STORY] Story and population size Travis Nixon
- [STORY] Story and population size Wells, Thomas
- [STORY] Story and population size Matt Mihaly
- [STORY] Story and population size John Buehler
- [STORY] Story and population size Matt Mihaly
- [STORY] Story and population size Jeff Cole
- [STORY] Story and population size John Buehler
- [STORY] Story and population size Jeff Cole
- [STORY] Story and population size John Buehler
- [STORY] Story and population size Michael Tresca
- [STORY] Story and population size Matt Mihaly
- [STORY] Story and population size Dave Rickey
- [STORY] Story and population size Jeff Cole
- [STORY] Story and population size Dave Rickey
- [STORY] Story and population size Paul Schwanz
- [STORY] Story and population size Daniel.Harman@barclayscapital.com
- [STORY] Story and population size Andrew Hefford
- [STORY] Story and population size Bryan "Cyngon" Helmkamp
- [STORY] Story and population size Bobby Martin
- [STORY] Story and population size Marian Griffith
- [STORY] Story and population size Bobby Martin
- [STORY] Story and population size Bruce Mitchener
- [STORY] Story and population size Sellers, Mike
- [STORY] Story and population size Bobby Martin
- [GENETICS] Evolving prey populations Adam Martin
- Stories, why? Ian Collyer
- Stories, why? amanda@alfar.com
- Purchase of Meridian 59 by Near Death Studios Brian 'Psychochild' Green
- Purchase of Meridian 59 by Near Death Studios Rob Ellis II
- Purchase of Meridian 59 by Near Death Studios Lee Sheldon
- Purchase of Meridian 59 by Near Death Studios Sellers, Mike
- Community Building Sanvean
- doesNotUnderstand behavior in C++ (was: Spoofs) Eli Stevens
- Importance of player roles Sasha Hart
- RTS aspects in MUDs (LONG) lhulbert@hotmail.com
- RTS aspects in MUDs (LONG) John Robert Arras
- RTS aspects in MUDs (LONG) Leland Hulbert II
- RTS aspects in MUDs (LONG) holding99@mindspring.com
- RTS aspects in MUDs (LONG) Dave Kennerly
- RTS aspects in MUDs (LONG) Takis Kalogiratos
- Agora, a Wiki for MUD topics Bruce Mitchener