Sixteen Examples of Entries
The first four items are the winning entries in 2006, abridged to save space. The remaining twelve items are summaries of examples from 2007. CM = Certificate of Merit. HM = Honourable Mention.
Winning entries for 2006
Feng Shui (HM)
Claim: The Feng Shui claim investigated by our group states that by attaching or including an image relating to a positive aspect of life to a person's workspace, the life of the individual will improve in this area.
Our Hypothesis: We believed that our survey would show results irrelevant to the image we attached to the school desk of the individual. Whether they see the image beforehand or not, we thought it most likely that a simple image stuck to a desk would not influence their life.
Results: The results supported our hypothesis. We distributed a survey throughout the class asking whether their life had improved, worsened, or not changed, and in what aspect. We repeated this twice more afterwards, this time including images. In one of the two repeats, the individual would not see the image of the certain life aspect. In the final trial, the image would be revealed. For each of the runs of the experiment, the same survey was given to each person. The results showed that, whether an aspect of their life had altered or not, the image attached to each desk played no influence. Our results therefore proved to be random, and indeed, irrelevant to the image we attached to each persons desk. Therefore this particular practise of Feng Shui would not be an accurate way to enhance an aspect of life. It may help in creating a motivation. However, as our experiment has concluded, it does not improve luck and aspects of the life of an individual. About 2300 words, 1 page of pictures, 11 references.
Zodiacs: Believe it or Not! (HM)
We are going to test whether Zodiacs are true by surveying a willing Cancer subject [their teacher?] for two half-days. The horoscope will be taken from The West Australian.
Day 1: Average Day. Feeling of boredom or dissatisfaction can be blamed on your ruling Moon. Today marks the end of a month long phase and from tomorrow onwards you can expect more variety, greater contentment and positive feedback. We started observing the subject from 9:00am to 12:30pm. We could not see anything in the prediction that really stood out as something to watch for, something to really prove our point. However, we questioned our subject on whether they had been feeling bored or dissatisfied and they answered no!
Day 2: Positive Day. Moon in your signs gives the impression of vulnerability, but inner strength keeps you resilient. Latent talents or desires could surface soon and others will be stunned as the layers of your personality are peeled away. We observed the subject from 11:30am to 3:23pm. Again, nothing really stood out in the horoscope prediction but we thought that the hidden talents might begin to show. However, no new talents were shown and we could not see other people stunned by our subject's personality. It was disappointing that the star signs didn't put up more of a fight and show us some real magic but that's that!
From our study, we came to the conclusion that the horoscopes don't predict the truth. It's probably just somebody sitting in an office, with the job of pulling twelve random predictions out of nowhere. We found nothing except for the fact that some horoscopes don't actually predict anything. Weird, hey! About 1000 words, 0 references.
I Ching (CM)
Summary: Over a period of around one month, research was conducted to test the accuracy of the predictions made by I Ching, a method of divination that originated in Ancient China. The readings it gives are decided by chance, or for those who believe in it, by synchronicity. The analysis of results revealed that for the group surveyed, I Ching did not prove to be accurate for a majority. Those who believed in divination were more likely to respond that their I Ching reading was more accurate.
We also conducted searches [too lengthy to be described here] into what others have already contributed to the testing of the I Ching. A large part of our research effort went into investigating synchronicity and the challenge of that phenomenon. It was concluded that a challenge to the validity of synchronicity would also be a challenge to the validity of I Ching. However the proofs against appear to be much stronger than the arguments for. Coincidence can be proved to be nothing amazing and especially meaningful (that is, not a case of synchronicity), as they are inevitable and simply the human mind making life interesting by looking for patterns.
Background: I Ching is the Book of Changes that is one of the most ancient and important literary works in the history of China. There are a total of 64 I Ching hexagrams (the form of the readings is in six lines, known as hexagrams) created by the throwing of coins or yarrow stalks. These hexagrams are supposedly mathematically based patterns that represent all the structure and changes that occur in our universe. Each of the I Ching hexagrams consists of a few verses of text that are used to predict the future. These verses are often mystical sounding and have a far deeper and multiple meaning than the words that are presented on the paper. Some people believe that the author of this book was Fu His, the first emperor of China in times as early at 3322 BC.
Synchronicity: According to Carl Jung, synchronicity is a principle where "no causal connection can be demonstrated between two events, but where a meaningful relationship nevertheless exists between them, a wholly different type of principle is likely to be operating". He believed that if one event occurs, another one will coincidentally happen as a result of the first event. It is the idea that coincidence occurs because of some "greater force" or because of a greater pattern in the world.
I Ching and Society: During our survey we found that nearly no-one had ever used the method of I Ching before or even heard of it. Although Australia is greatly influenced by different cultures from all over the world, it was obvious that I Ching has not had a significant influence on the lifestyles of people here. Although in general I Ching has little influence on our society it does make up an important part of some people's lives. As with any other mystic revelation, I Ching does have its cult followers. We came across many stories of people who believe I Ching has the ability to accurately predict the future and to make the right decisions for them, including a masters' thesis that tested and calculated (albeit not very scientifically) the accuracy of I Ching for divination: Granillo, Tony (1998) Exploring Prediction and Meaning through the 1 Ching Oracle, Antioch University, Seattle.
Method: We surveyed a number of people, mostly girls aged 13-15 [seemingly N = 21]. We began by getting them to throw three 10 cent coins, which were used to determine their I Ching hexagram. But their actual reading from the Book of Change would not be looked at until the next week to ensure that our investigation was as objective as possible. After taking down the reading, we got the people to fill in their answers to a set of 4 questions using a 5-point Likert Scale. One week later, we revisited these people and requested that they tell us the high points, low points and major decisions of their past week. We then gave them a reading that was based upon the hexagram that was made the previous week from their coin throwing. However, for some of the people, we gave them fake readings. We then asked them to rate their reading for accuracy on a scale of one to five. The Book of Change used in this investigation was: Wang, Ronpei & Ren, Xiuhua (Translators) (1993), Book of Change, Shanghai Foreign Language Education Press, Shanghai.
Results of Survey: [The original 5-point scales have been collapsed to
Discussion: As we only gathered results from a group comprising mostly girls aged 13 to 25 years old, our results cannot be justifiably applied to every case in general. Our first four questions found that many people did not consult divination regularly and that most did not believe it to be particularly accurate in prediction. For question 5, 38% of surveyees felt that their reading was completely inaccurate, and 19% felt it was fairly inaccurate. On the other hand, 19% felt it was fairly accurate in its prediction of their past week, however none felt that it was a miraculous perfect match. From this we conclude that I Ching itself is not an accurate divination method (indeed, it may be that there is not such thing as an accurate divination method). We soon gathered that there was a relationship between how much the surveyee believed in I Ching and divination and whether they thought the reading was accurate. We noted that all those who felt their reading was completely inaccurate were generally strong disbelievers and non-users.
Examples of the responses to fake readings: One surveyee said she disbelieved in divination, but said that predictions had sometimes read true for her previously. She rated her reading as not close but not wrong either. Another said that she strongly disbelieved in divination, but said that predictions had sometimes read true for her previously. She rated her reading as a "pretty close match", although it was false. These results, along with other results, give evidence that it is the belief in the divination that makes the divination "accurate" in their prediction. Some disbelievers commented that if they believed in divination, they could see how the reading could possibly relate to what had happened to them, but felt that in reality the readings were sham.
Conclusion: We found that I Ching was not consistent in accurate prediction of the future, and that whether or not it was "accurate" depended upon the view of the person about divination, that is, if they believed in the accuracy of divination and that it worked, it was more likely that they would say that their reading would be accurate. The human mind likes to find meaning and pattern in its environment and so remembers the patterns that do occur and forgets the parts where the patterns fall apart. On the other hand, our research has not a large enough sample size or range in order to completely prove or disprove whether I Ching is accurate, but the trends we have seen lean towards a "maybe not". Carl Jung said in his Foreword to Lectures on the I Ching "In the I Ching, the only criterion of the validity of synchronicity is the observer's opinion that the text of the hexagram amounts to a true rendering of his psychic condition"'. We have discovered that in "the observer's opinion" the "text of the hexagram" did not amount "to a true rendering of his psychic condition", so our results on I Ching also dispute the validity of synchronicity. About 7200 words, 3 pages of graphs, 13 references.Bibliography
Moran E & Yu J (2002). The Complete Idiot's Guide to I Ching, Pearson Education Inc, USA, pp 64-65.
Jung CG (1986). Foreword to Lectures on the I Ching; Constancy and Change, Princeton University Press.
Pearson CL (2002). Consider the Butterfly, Gibbs Smith Publisher, Hong Kong.
Eastaway R & Wyndham J (2003). Why Do Buses Come in Threes? Chysalis Books Group, London, pp 51-54.
Gilovich T (1991). How We Know What Isn't So -- The fallibility of human reason in everyday life.
The Free Press, New York, pp 174-179.
Peat FD (1987). Synchronicity: The Bridge Between Matter and Mind, Bantam Books.
Jung CG (1970). The Collected Works of CG Jung, Volume 8, Princeton University Press.
Wang R & Ren X (Translators) (1993). Book of Change, Shanghai Foreign Language Education Press, Shanghai.
I Ching divination (no date). Wikipedia. Retrieved 26 May 2006 from http://www.answers.com/topic/i-chinq-divination
Traditional Methods (2004). Casting I Ching Hexagrams Retrieved 26 May 2006 from http://www.dentato.it/iching/
Roth RF (2002). Introduction to Carl G Jung's Principle of Synchronicity
Retrieved 27 May 2006 from http://www.psychovision.ch/synw/synchronicity_junq.htm
Likert scale (2006). Wikipedia: The Free Encyclopedia. Retrieved 6 May 2006 from
The Mystery of Chance. Retrieved 15 May 2006 from http://www.strangemaq.com/mysteryofchance.htmi
Belief in Tarot Cards (CM)
Summary: The paranormal claim that we investigated was that younger teenagers aged 12-13 would have a higher belief in tarot cards than older teenagers aged 14 and older. We also thought the backgrounds of people would have an effect on their beliefs, European and Asian backgrounds would provide a higher rate of belief that the Australian background, who we think would be slightly more skeptical of tarot cards. We designed a survey to test our audience and their perceptions of tarot cards and how much faith they put in this paranormal claim. Results showed that the older teenage group had a higher belief in tarot cards than the younger ones, which went against our original expectation about belief in this paranormal claim.
Evidence For and Against: The way the cards are read differs according to which source you believe. One source says to simply "Come up with your own card meanings and descriptions to personalize your reading style. Don't be afraid to follow your instincts". [E-how, 1999] Which could mean one of two things. One is that the meanings are simply made up out of the readers own mind depending on what they feel like, with no real prediction or true answers being offered. The other is that the card interpretations actually come from psychic powers, so the cards are actually directing what you are interpreting. But the reading of the cards is very vague, it is even hard to define what tarot cards are supposed to tell you in the first place. "Assigning a specific meaning for tarot cards, across all decks and spreads, is quite impossible". [About, 2006] So how can we be sure about what they are telling us, and which ones are real?
On the internet, there is a blog from an author unknown [Kontroller X] who claims to have conducted an informal experiment by creating their own fake set of tarot cards with different meanings. They tested their cards on a small group of people with varying beliefs, and noted that most people did say that it was correct and the reading told them a lot about their lives.
A good example of the yes/no sides of tarot can be seen in an online forum where to people make comments to each other back and forth about tarot cards and how they could be tested. They relate tarot card reading to a game of 20 questions; which brings us to the conclusion that it is not actually telling the future, just cherry picking by the reader until they get the right answer. It's interesting to see that the cards are more of a medium and what they tell us isn't important. This leaves us with the question, "Then why bother with them?" [Doctor X, 2005]. What does make these cards so special if they are only pieces of paper?
In researching tarot cards, we looked at several sources in order to understand their origins and also the supposed effects and benefits of using them, as well as any counter-arguments to the effects and benefits. We also visited several sites that give some personal opinions about the tarot cards. We found out that some of the sites tend to believe in the cards whilst some do not.
Our Study: Aim: To investigate the widespread belief of tarot cards based on factors of ethnicity and age. Procedure: (1) The target audience of Year 8, 10 and 12 students and some teaching staff was decided. The total number of people tested being 100. (2) A survey to test the target audience's awareness and belief in tarot cards was designed. (3) The surveys were distributed. (4) The results were recorded. Hypotheses: (1) The younger age group of 12-13 years will be more believing in the tarot cards, and the older age groups less believing. (2) Europeans and Asians will be more believing, and Australians less believing.
Results as a percentage of N
Limitations of the Survey: Sample could have been larger and more varied, allowing more control of variables. The Year 11's and 12's were having exams at the time and were difficult to get hold of, thus limiting the target audience across the high school. The sample chosen could have had an impact on the survey due to the standardised environment of which they came from (Methodist Ladies' College).
Conclusion: Hypothesis (1) was not confirmed. People in the age group of 14-16 years proved to be the most believing group. Hypothesis (2) was confirmed. Europeans proved to be the sample group more believing in the accuracy of tarot cards, probably due to the country of origin of tarot, which is fathomed to be a country in Europe. Furthermore their cultural background consists of mystical phenomena such as tarot cards. Australians proved to be the most disbelieving group. This could be due to their cosmopolitan lifestyle, which has no room for mystical beings. Asians were the most unsure group. This could be due to clashing of tarot cards with other Asian mystical beliefs consisting of anything from black magic to feng shui. About 4200 words, half a page of pictures, 2 pages of graphs, 10 references.Bibliography
Dragonlady and Doctor X (2005) Tarot and Randi's Challenge
Retrieved 22 May 2006 from http://realitycoffeeandhumblepie.com/smf/index.php?topic=134.0
Kontroller X (2006) Are Tarot Readers full of Crap or what?
Retrieved 21 May 2006 from http://blogs.ign.com/KontrollerX/2006/04/12/11426/#comments
Unknown (about 1999) How to read Tarot Cards
Retrieved 21 May 2006 from http://www.ehow.com/how 17446 read-tarot-cards.html
Unknown (about 2006) Reading Tarot Cards).
Retrieved Sunday 28th May 2006 from http://healing.about.com/cs/uc directory/a/uc readtarot 3.htm
Bodet N. Tarot Cards by Nicolas Bodet world of playing cards. Retrieved 30 May 2006 http://www.wopc.co.uk/downloads/BodetTarot.jpg
Kelly L (2004). The Skeptic's Guide to the Paranormal. Allen and Unwin, Crows Nest NSW.
Shermer M (2002). Why People Believe Weird Things --
Pseudoscience, superstition and other confusions of our time. Owl Books, NY USA.
Carroll RT (2003). The Skeptic's Dictionary --
A collection of amusing deceptions and dangerous delusions. John Wiley, USA.
Examples from 2007
The entries for 2007 showed good levels of critical thinking and good awareness of where claims can be deceptive. Altogether a very good effort. But why did some entries succeed while others did not? Below are critical summaries (average 200 words) prepared by WA Skeptics that cover everything of interest in each of 12 entries. Half succeeded and half did not. Can you see why? Hint: success is unrelated to length of entry or to number of co-authors. Year 9 entries are listed before year 11 entries. Within each year the entries are listed A-Z by topic.
Astrology and Zodiac Signs
Summary: Review misses the many hundreds of existing tests of astrologers and their claims even though they are detailed on one of the cited websites (www.astrology-and-science.com), which led to wrongly stating "a conclusion about whether astrology works has not been reached" and "astrology is difficult to test". Surveyed 50 students (10 each from years 8-12) about their knowledge of, and belief in, astrology. Overall responses were:
Astrologers can predict personality and events 26% yes 58% no 16%
As a follow-up to the last question, say why. Those answering yes said because astrology was fun, has been around a long time, their friends believe in it, it seems to work, and the media encourages belief. Those answering no said because astrology was weird, implausible, vague, contradictory, and for entertainment only. Nobody mentioned the extensive scientific evidence against astrology. About 8400 words, 3 pages of graphs, 10 references.
Crystal Therapy (HM)
Summary: Entry form says year 9 but title page says year 10. Gave a quiz about crystals (which included trick questions) to five science teachers and five paranormalists (not clear who these were) recruited from local markets and new age shops. Do their views differ? Yes:
Have you hard of crystal therapy? ST 80% yes. PN 100% yes.
Reasons were listed for believing in crystal therapy, mostly involving various "energies", most of them endorsed by the paranormalists including a fake reason involving non-existent "resculent energies". It was predicted that the views of science teachers and paranormalists would differ, the former being disbelievers and the latter being believers, which was confirmed by the results. However, none of the science teachers had tried using crystals, nor had two of the paranormalists even though they were believers, which suggests that controlled tests using actual crystals are needed (the few published tests have been uniformly negative).. The authors started out disbelieving and ended up just as disbelieving. Their conclusion is that the science teachers' views are the more correct. About 2700 words, 3 pages of graphs, 7 references.
Summary: Can graphology really reveal your personality? Review of the supporting evidence cites the American Psychologist (should be Psychological) Association as finding "that graphology can be a reliable tool", when all that happened was the presentation of some unspecified findings at an APA conference. (No reputable textbook of psychology treats graphology with anything but disdain, a point confirmed by remarks under "Evidence Against Graphology", except the references there are not listed and are therefore impossible to follow up.) Tested by comparing extraversion scores based on a 10-item test with a graphological extraversion score based on ten handwriting features said to be related to extraversion such as large writing, right slant, and heavy pressure. Results for nine students in year 8 showed that graphology predicted far fewer extraverts than the personality test. Results for nine students in years 11-12 were similar but less extreme. About 5200 words, 4 pages of graphs, 11 references.
Unfortunately this and other entries did not make more of their results by calculating the correlation between results of interest such as the answers to A vs the answers to B, or in this case test scores vs graphology scores. Correlations are easily calculated with any scientific hand calculator (just $3.99, not a misprint, at Woolworths in 2007) and are therefore within reach of any student even though formal teaching in the concept may be some years away. In this case the correlation between scores was -0.07 on a scale of -1 = perfect inverse correlation through 0 = no correlation to +1 = perfect correlation, which confirms the conclusion.
Murphy's Law and the Mystery of the Toast
Summary: Gave ten examples of Murphy's law (whatever can go wrong will go wrong) to a total of 91 students (24 from year 7, 24 from year 8, 20 from year 9 who were doing Skeptic Awards assignments, and 23 from year 9 who weren't) who had to indicate which ones they agreed with. Overall 41% had not heard of Murphy's Law. Year 7 were more believing, as (surprisingly) were year 9 doing the Awards compared with year 9 not doing the Awards. The five most popular examples were:
Toast always lands butter side down.
The most common sources of information about Murphy's Law were fathers, followed by the media and friends. The authors experienced Murphy's Law for themselves when they were wanting to finish their calculations in the library on the last day of school, when the early closure of the library (due to staff illness) was announced. About 2500 words, 12 pages of graphs, 3 references.
Summary: Review missed the (admittedly hard to find) scientific evidence in which tests of palmists had failed to support their claims. Tested by photocopying the palms of 6 science teachers (only 5 were usable) and comparing their self-reported personality, health and hobbies via an 11-item quiz with that derived from their hand features (lines, shape, etc) as given on two palmistry websites. The websites often disagreed and neither was accurate, one averaging only 55% agreement with the teachers' responses and the other only 29%, which suggests that the success claimed by palmists is largely due to cold reading. Future tests could be improved by involving actual palmists (as was done in the missed evidence). About 2200 words, 1 page of graphs, 10 references.
Summary: Surveyed 30 males (ten each from years 8, 10, and 12 recruited via the authors' brothers), 30 females (ten each of years 8, 10, 12), and 20 teachers (ten male and ten female) to see if age and gender influenced belief. Results showed a consistent gender difference as follows (recalculated from the original results, which were not well presented):
What is a psychic experience? Seeing future, accurate readings
The above psychic reading was the original set of Barnum statements due to Bertram Forer in 1949 (he assembled them largely from a newsstand astrology book), who found that most people rated the set as a very accurate description of themselves. The lower ratings obtained here were probably due to it being presented as "a famous psychic reading" rather than as a personal reading. Without exception, females in each age group were more believing than males. As predicted, the most believing (70% yes) were the youngest females and the least believing (20% yes) were the adult males. Problems: some students did not have enough time to complete the survey and had to be re-surveyed, male students did not always take the survey seriously, teachers resisted revealing their age, and question 4 (what had the biggest influence on your belief in psychic powers?) was confusing to those who did not believe. About 6100 words, 5 pages of graphs, 6 references.
Summary: Review missed the (admittedly hard to find) scientific evidence in which tests of tarot readers had failed to support their claims. Surveyed 15 each of year 8 students, year 12 students, and teachers, all at the same school, about their belief in tarot cards. The biggest differences were:
Do you believe in tarot cards? 60% yes for year 8 13% for
Overall, year 8 were the most believing, teachers the least believing, which supports Michael Shermer's theory that the more educated a person the less likely they are to believe in weird things. One problem was people making their own boxes to tick. Another was the lack of time due to competition from holidays and music lessons. But both authors enjoyed the project and learnt a lot. About 2600 words, 4 pages of graphs, 6 references.
Year 11 entries
Summary: Review includes much irrelevant information (eg on astrological houses) and misses the many existing tests even though they are documented in one of the cited references (Sun Sign Astrology in The Skeptic Encyclopedia of Pseudoscience). Sun signs were tested by giving 24 students in year 10 (two per sun sign) three sets of 12 unidentified and jumbled-up daily horoscopes for the previous day (25 September 2006), each set from a different website, from which they had to choose the one that fitted them best. Two conflicting versions of the test are given making it difficult to tell what was done, but the tabled results indicate there were 8 hits vs 6 expected by chance, or 4 hits when the signs were shifted two places back (so Cancer was now labelled Taurus) as a test of precession. Both results indicate that accuracy was no better than guessing. Choosing was difficult: "There are four here that apply to me" said one student. "They are all vague and could apply to anyone, so how am I supposed to choose?" said another. Also gave 30 students in each of years 9-11 a quiz to check their belief in astrology. Year 9 tended to be the more believing. Overall responses were:
I plan my day according to star signs 8% yes 69% no 23% unsure
Innovations included shifting the signs to test for precession (seldom done in tests like these), offering chocolate frogs as a reward for completing the test, and including boys (via one author's boyfriend) as a check on gender effects (except they broke up before it could happen). About 7700 words of text, 10 pages of graphs, 6 references.
Cold Reading (HM)
Summary: Review includes a critique of John Edwards and his TV programme Crossing Over. Made a short video of a (faked) psychic reading and showed it to a total of 17 students (5 from year 8, 2 from year 9, 10 from year 10), who then completed a quiz about the video. Overall responses were:
Was it a documentary or psychic reading? Most said the latter.
About 3600 words, 5 pages of graphs, 10 references.
This and other entries usually reported trends in their survey responses due to age etc (in this case the authors predicted that belief would decrease with age, but year 10 turned out to be more believing than years 8 or 9). However, sample numbers were nearly always too small for the reported trends to be meaningful, in which case a statement to that effect appears in these abstracts rather than the reported trends.
Hair Colour Stereotypes (CM)
Summary: The stereotype says blonde = dumb. Tested by giving an IQ test (disguised as a personality test, the higher the score the higher the IQ) to 40 students in years 9 and 10. In year 9 the average IQ score for 12 blonde students was 5.96 vs 7.00 for 8 brown/black-haired students (5.96 is from the text, the graph shows about 6.5), in favour of the stereotype. In year 10 the averages for 8 blonde students and 12 brown/black-haired students were 9.25 vs 8.79, this time against the stereotype. For years 9 and 10 combined the averages were 7.43 vs 7.90, at first sight slightly in favour of the stereotype. However, the small samples and the unreliability of measuring IQ with a short test (as here) support the conclusion that there is no real difference. So the stereotype was not confirmed. (Inspection of the results suggests that the standard deviation of each average is likely to be around 3, far too large to give the 0.47 difference any significance, which again supports the conclusion.) About 4600 words, 3 pages of graphs, 16 references.
Moon Landing Hoax
Summary: Thoroughly reviews published views for and against even though based on only three sources. Gave 52 students from year 10 eight questions about space travel and moon landings. All believed that people had landed on the moon, but some were not sure the landings included Apollo 11:
Humans have landed on the moon 100% yes
The author originally believed in the hoax but was persuaded to change her mind by the evidence she had collected. About 6800 words, 1 page of graphs, 3 references.
Urban Legends (HM)
Summary: Eight urban legends (eg the youngest mother on record is a five-year-old Peruvian girl, St Bernard dogs are bred for food in China) were given to 25 girls from year 10 and 25 boys from year 10 at another school recruited via brothers of the authors. Were girls more believing in these legends than boys? The results showed no difference:
Girls 45% believers, 30% unbelievers, 24% unsure.
Non-religious students tended to be more believing than Christian students, while Jewish and Muslim tended to be less believing, but the sample numbers were too small for these differences to be taken seriously. About 5100 words, 4 pages of graphs, 17 references.