Blazing the Trail 1 - Perseverance

I am currently reading Blazingthe Trail: Essays by Leading Women in Science edited by Emma Ideal and Rhiannon Meharchand. The inspiration for this book came at the Fourth IUPAP International Conference on Women in Physics in South Africa in April 2011. At that conference there was a presentation about the book  Lilivati’s Daughters: the Women Scientists of India, which is a collection of essays about female physicists in India. Ideal and Meharchand decided to produce a similar book about women physicists in the United States. Blazing the Trail consists of thirty-five short essays by women working in physics or related areas. Rather than try to review the whole book I am going to record my thoughts as I read.

The first thought to strike me was the importance of perseverance. The author of the first essay, Christine Aidala, describes how she started graduate school immediately after finishing her bachelor’s degree but dropped out quite quickly and travelled for a while. Eventually she decided she wanted to get back into research. She tried to find a suitable position using her contacts from her undergraduate days without any luck but eventually found a suitable research position, completed a Ph.D. and is now in a tenure-track position in a university. There are two points here. First, you do not have to follow one route into research. You can recover from apparent disasters. Second, when you are looking for a position the chances are that most of the opportunities you pursue will not work out. Keep trying. 

All it takes is hard work?

Recently I had an interesting juxtaposition  of experiences.

The first was attending a meeting at which some women scientists, all in academia, spoke. They all agreed that hard work, being passionate about what you do and following your dream were the keys to success. I felt a bit uncomfortable with that as I know many women who worked hard, were passionate about what they did and followed their dream and are no longer scientists, at least in the sense of being actively engaged in research. During questions at the end they were asked if women are at a disadvantage when it comes to careers in science. 'No, no' they all agreed, ignoring the accumulated data. In fact, they unanimously claimed, if anything being a woman was an advantage, since women are more visible.

The other was reading Claude Steele's book WhistlingVivaldi: And Other Clues to How Stereotypes Affect Us. (The title comes from the experience of a young black man. He found that people often reacted to him with fear until he started whistling classical music. The act of whistling classical music was enough to replace the stereotype 'violent young black man' with the stereotype 'educated individual'.) Claude Steele is the discoverer of the phenomenon of stereotype threat, in which being a member of a group that is stereotypically thought to be worse at some task can reduce performance on that task. One of the phenomena he describes in Whistling Vivaldi is what he calls 'over-efforting'. Essentially this describes students who fail to use effective strategies, presumably from a desire to prove themselves as individuals. One example he discusses is a notoriously difficult course that was required for entry into medical school. Many students would audit the course in one year and take if for credit the next or find ways of substituting a different course. However, while white and Asian students were happy to use these strategies, many black students would insist on taking on the course for credit even when, from the point of view of getting into medical school, it would have been strategically better to withdraw and either take it for credit the following year or substitute a different course. This led Steele, who is black, to question the advice he received from his parents that the key to success was hard work. He goes on to discuss an interesting experiment performed by O'Brien and Crandall (http://psp.sagepub.com/content/29/6/782.short). A standard way of testing for stereotype threat is to have two groups sit a test. For one group membership of a particular group is made relevant; for the other, membership of that group is minimised. For example, if a group for which gender has been made relevant sit a maths test then the women tend to do worse that the men whereas if the group has been told that gender is not relevant then men and women perform the same. What O'Brien and Crandall did was have the groups do an easy maths test and a difficult maths test. Making gender relevant actually improved women's performance relative to men on the easy maths test but made women's performance worse on the difficult maths test. Steele suggests that responding to challenges by working hard is a successful strategy up to a point but fails when the challenges become more difficult.

The question this juxtaposition of experiences raised in my mind was: what should the message we give young women be? Should it be 'Work hard and you can achieve anything'? Or, should it be 'These are the potential difficulties and here are some strategies to get round them'?

Evaluating Women in STEM Interventions

When I started working for WiSETI in Cambridge in 2003 I was employed to work on a project specifically aimed at increasing the number of women applying for positions in science and engineering. At that time I thought that evaluating such a project would be easy. We knew what the proportion of applicants for lecturing positions who were female was before the project started. All we would need to do would be to monitor the proportion of applicants who were female during and after the project and see if it increased. As the project progressed I came to see this expectation as being very naïve.

The problems are:

1. Small numbers. How do you know whether an observed increase is due to the intervention or just a random fluctuation?
2. If you aggregate data from different departments are you doing it in a meaningful way? You expect more applications from women for a position in Botany than for one in Computer Science. It could even be that some areas within a subject have more women than others.
3. Measures are not evenly applied across departments. Some departments are enthusiastic and follow advice, some are enthusiastic but do their own thing, some are not enthusiastic but go through the motions and some are not enthusiastic and ignore advice. (And these descriptions are points along a continuum, not categories.)
4. Other relevant variables change during the course of the project – legislation changes, policies change, heads of department and departmental administrators retire and are replaced, and new nurseries open. It is impossible to be sure that you are comparing apples with apples.
5. Is the proportion of job applicants who are female even the right quantity to monitor? What matters in the end is how many women are appointed. Does increasing the number of female applicants from say 5 out of 35 to say 10 out of 40 actually increase the likelihood that a woman is appointed? Or does it just mean that five extra women have devoted a considerable amount of their precious time to writing a job application and another group of people have had to spend their time reading them? If relatively few women are applying for lecturing positions in STEM does this mean that women are establishing themselves as independent researchers and then not applying for academic positions or does it mean that they are not establishing themselves as independent researchers and hence qualifying for the academic positions in the first place? Is it that women lack confidence and hence don’t apply for positions? Or is it that women tend to be time poor and therefore less likely to spend some of that precious time applying for a job unless they think they have a reasonable chance of success? If women are less likely to apply for a job for which they are qualified than men are, is the solution necessarily to persuade women to behave more like men?

So, even a project with relatively well-defined objectives is not necessarily straightforward to evaluate.

If you are trying to decide how to most effectively deploy available resources to achieve the best effect then you also need to take into account what resources were devoted to the project. When you are estimating the resources used by a project do you take into account time effectively donated to the project? How do you compare a project that has a positive effect on a small fraction of participants but can be delivered to a large number of people with a project that makes a difference to most of its participants but can be delivered to only a few people?

There are a lot of questions here. I would really like to see some rigorous discussion of how interventions are evaluated, provided, of course, that this is not at the expense of actually doing something.

Motherhood and Career

Over Christmas, one of the books I read was ‘Reading Women: how the great books of feminism changed my life’ by Stephanie Staal. Staal grew up taking it for granted that women could have professional careers. She went to college, worked and then did a Masters degree in journalism before working as a reporter and writing a book. The she got married and had a baby. She took a pragmatic decision to switch to freelance work, taking into account her working hours as a journalist and the costs of childcare. Eventually as she struggled to find the time and the energy to work she started to feel that she was losing her identity as a professional with a career. Then one day she was in a book shop and picked-up Betty Friedan’s ‘The Feminine Mystique’, which was published in 1963 and helped spark the feminist movements of the 1960’s and 1970’s. She found that she now understood what Friedan was writing about in a way that she did not when she read the book an undergraduate. Eventually she decided to repeat ‘Feminist Texts’, an introductory survey of the major feminist works, at her alma mater, Barnard College, in New York, in order to see whether  they would help her find a way forward.

The book interweaves her thoughts as she reads through and discusses the texts with her own experiences and, if nothing else, provides a quick introduction to feminist thought over the centuries.

A passage that particularly caught my attention was in a discussion of Virginia Woolf’s A Room of One’s Own. Virginia Woolf famously proposed that in order to engage in creative activities like writing a woman needs a room of her own and a personal income of £500 a year. Staal writes:

Crowded into an urban apartment, working to regain my professional footing, keeping watch of a young child, I thought about Virginia Woolf’s conditions for female creativity. If I had the money – five hundred pounds converted into U.S. currency and adjusted for inflation, of course – and a room of my own, which I sort of had here by the kitchen, was that really all it would take? My daughter pounded on the door of my home office. “Mommy? Mommy? Mommy?” I stopped typing. The words of the sentence I had been writing scattered, the muse frightened away. I swung open the door in a motion of irritation. And there stood my daughter, holding out to me a piece of paper with a poem she had written, her expression serious and proud.“Mommy,” she said, “I wrote this for you.”

That seemed to me to sum up the dilemma that women face one we have children. Yes, we want to use our intelligence and education but we also want to have time to spend with our children. I do not believe that there is a universal right answer. It depends on your health, the health of your children, the support available from family or networks of friends, cultural and religious background, how much you earn, how much your partner earns, whether childcare is available and many other things. But, if organisations seriously want to retain women then they need to support whatever works for individual women, whether it is working full time (depends on access to childcare, school holiday programmes, after-school programmes), working part time (without prejudice to career prospects) or taking a career break (depends on support for returners). This should not be seen as a luxury but as essential for retaining the best people.

I also think we who have experience of combining career and children also need to be more willing to talk about what it is really like. Not just the constantly feeling that whatever it is you are doing you ought to be doing something else but the feeling that an identity that has been important to you as a professional is becoming submerged.

Guide and Inspire

I am back blogging again after a break.

Last week I arrived back from a trip to the UK and in the pack of held mail left by the Post Office I found my November Toastmasters magazine. In it was an article called ‘Dare to Delegate’ by Judith E. Pearson. The first thing that struck me in this article was the sentence

‘Since Toastmasters leaders cannot hire or fire, they must instead guide and inspire.’

Toastmasters is a voluntary organisation so indeed leaders cannot hire and fire but many universities are in a similar position in that it is difficult to compel academics to do anything.

The second is the importance of appealing to people’s needs and values.

Some academics may see gender equality as an important value in itself, consistent with a commitment to fairness, others may see it as benefitting their institution, and there are some for whom the only effective appeal is to self-interest, for example, not following the rules can lead to wasting time, energy and money in sorting out the consequent mess.

You do not have to be in a recognised leadership position to promote gender equality but if you want to be effective then “guide and inspire” and tailor your message to the recipients actual needs and values and not to what you think they ought to be.

Transforming Organizations

In a previous post I wrote about Teresa Rees’s description of three approaches to gender equality: tinkering, tailoring and transforming. Transforming is the process of modifying structures and processes to embrace diversity. Embedding change frequently requires us to address the unspoken assumptions and beliefs that underlay previous behaviour. This process is sometimes called ‘culture change’. There are layers of culture (see, for example, Cathy Trower’s presentation at the Third Annual Georgia Tech NSF ADVANCE Conference 2004:

-        Structures and processes, with the additional complication that what actually happens may differ from what is laid down in an organisation’s policies and procedures.

-        Rhetoric – what people say.

-        Underlying assumptions and beliefs.

The first step in transforming an organisation is to reflect on what sort of organisation you have. An initiative that may have been very successful in one organisation may be a complete flop in another. In Understanding Organizations Charles Handy identifies four cultures: power, role, task and people oriented. This is not a unique way of classifying organizations but it has the merit of being simple and easy to relate to personal experience. Each culture has its strengths and weaknesses. Whether a culture is appropriate or not depends on the environment the organisation is in.

The power culture depends on a small group or a single person who controls a central source of power.  Organizations with this culture can react quickly but may move in the wrong direction. They are often vulnerable to the loss of a key individual. These organisations tend to have few rules and procedures. According to Handy an organisation of this type ‘depends on trust and empathy for its effectiveness and on telepathy and personal conversation for communication.’

The role culture is more commonly referred to as a bureaucracy. Bureaucracy has negative connotations. As Handy notes: “No one, it seems, approves of bureaucracy except, interestingly, lots of people in organisations who like to know where they stand, what they have to do, who is in charge and what the rules are.” Role cultures are characterized by procedures, e.g. job descriptions and procedures for communications, and rules. Such organizations do well in stable environments but are slow to react to change. They tend to value roles and processes over the individuals who perform them but do offer security and predictability.

The task culture is focussed on completing a particular job or task. The strategy is to bring together teams of people with appropriate expertise and resources to accomplish a specific task. Individuals have control over their work and are judged by results. Overall control largely lies in allocating projects and resources as day-to day decisions have to be made by the individuals concerned. Organisations of this type tend to be flexible.

In the person culture the individual is central, as Handy puts it: “If there is a structure or an organization it exists only to serve and assist the individuals within it.” Handy’s metaphor for its structure is “a galaxy of individual stars”. Although few organizations retain such a culture since organizational goals are eventually imposed on individuals, there are individual people with this orientation, usually people with highly specialised expertise. Handy notes that such people “often feel little allegiance to the organization but regard it rather as a place to do their thing with some accruing benefit to the main employer.” Managing such individuals is not easy as many of the sources of power are ineffective.

The point of looking at a classification such as this is not to shoehorn any particular organisation into one particular category but to provide a basis for understanding what ways of influencing people are likely to be effective. To people in a role culture the fact that someone perceived to be in authority has made up a rule that something should be done is seen as sufficient reason for doing it. To someone who is oriented towards a person culture the fact that someone generally perceived to be in authority has made up a rule that something should be done may be seen as irrelevant. What are the preferred means of communication? There is little point in sending out a memo if people prefer personal communication.

Tensions can arise within organizations such as universities when different groups have different cultures. University bureaucracies (e.g. finance, human resources) inevitably tend to be bureaucratic while many academics incline to the person or ‘galaxy of individual stars’ culture. Individual research groups may have a power culture with the PI who brings in the money exercising the power. Each of these cultures has their own, usually unspoken, underlying beliefs. Efforts to bring about change in the way a university handles diversity fail if they do not take these differing beliefs into account.

Inappropriate Behaviour

Recently Athene Donald blogged about two situations: a male colleague speaking to her on the phone told her that he could hear she was getting emotional and her experience at a reception after an event when a complete stranger put his arm around her waist when a photographer appeared. FemaleScienceProfessor also blogged about her experience as the only female member of a working group when a senior professor in the group mentioned, twice, that she was only included in the group as a gesture to diversity. FemaleScienceProfessor described her response, which was to ignore the remark and move the business of the group forward. There were a range of responses to these posts, many offering suggestions for ripostes in these situations, ranging from the manipulative to 'forgive and forget'.

I am from a cultural background that does not encourage physical contact. I think my own reaction to arm-draping behaviour would have depended on the stage of my career. As a very early career scientist I think my reaction would have been 'So this is what people do at this type of event'. That may also have been the conclusion drawn by my male collaegues. Later on, pre-assertiveness training, I think I would have thought something along the lines of 'I am not comfortable with this but I'll probably never see the guy again so why make a fuss'. After five years exhorting other people to use assertive responses I hope my response would be to move away, saying something like 'I would prefer you didn't do that. It makes me feel uncomfortable'. Assertive responses are not always appropriate. They rely on the person you are interacting with reacting rationally. If they are focussed on 'winning' the encounter this may not be the case. (The Harvard Business Review, May 2008, has an interesting study by Deepak Malhotr, Gillian Ku and J.Keith Murnighan on how competitive arousal leads to poor decision making.)

In general, how you respond to instances of remarks along the lines of 'you are getting emotional' or 'you are only on this committee because we needed a woman' depends on the context. If you suspect that the speaker is trying to provoke a reaction then it may well be the best strategy to ignore his remarks. A humorous remark, if you can think of one, may well be a good response if you think the speaker is just being thoughtless. 'I'll ignore that remark' may be a way of taking control without making too much fuss. Clearly, no one should be expected to object to every single belittling remark. There are circumstances in which it is better to stay focussed on the matter in hand.

In a comment on Athene's post, Dorothy Bishop mentioned 'an incident where everything hinged on whether or not a male academic had sleazy intent in an interaction with a female colleague', (more information on her blog).Here is a definition of inappropriate behaviour taken from the dignity@work policy of the University of Cambridge:

...behaviour is defined as inappropriate if it is:
Unwanted by the recipient.
Perceived by the recipient as violating his or her dignity and/or creating an intimidating, hostile, degrading, humiliating or offensive environment.
Having regard to all the circumstances, including the recipient's perception, the behaviour could reasonably be considered as having that effect.

There is nothing in this definition about the intentions of the person whose behaviour is in question. The focus is entirely on its effect on the recipient. Of course, the intentions of the person accused of inappropriate behaviour are relevant to what actions are taken to resolve the issue but they are not relevant to the question of whether inappropriate behaviour has, in fact, occurred. What sort of behaviour might be regarded as inappropriate? One scenario, from diversity awareness training material, involves a group of three people, two men and one woman. One of the men tells a joke that hinges on a comparison between a woman with pre-menstrual tension and a terrorist.(I don't remember the joke. It wasn't particularly funny, though I didn't think it was offensive either.) One of the implications of this definition of inappropriate behaviour is that there are bound to be grey areas where some people see behaviour as violating someone's dignity and others do not. This is why people should be encouraged to be able to say when someone's behaviour makes them feel uncomfortable and people who find they have inadvertently given offence to apologise and stop doing it rather than become defensive. No one wants to invoke their institution's formal dignity@work process every time someone makes an unfortunate remark.

What can institutions do to help? We should not be relying on the recipients of inappropriate behaviour or the efforts of a few individuals to address this issue. A common intervention is compulsory training. The problem with this option is that it can end up being counter-productive leading to a culture in which treating people with respect is perceived as enforced political correctness and making inappropriate behaviour more likely and harder for the recipients to deal with as they have to transgress group norms to do so. Macho 'zero tolerance' policies have the disadvantage that often no one is clear what it is that is not tolerated. Also, the problem tends to be framed  as identifying and punishing offenders, which can be a barrier to people seeking help with a difficult situation if what they want to achieve is a resolution that is acceptable to everyone.

Institutions should recognize that staff and students come from a variety of different backgrounds and encourage departments to find ways of promoting appropriate behaviour. It could be as simple as the PI of a group sitting down at morning tea and saying 'I heard about this incident' (making sure it is fictional or anonymised), asking for suggestions on how to deal with it and making sure people understand why someone might find a particular behaviour inappropriate in a professional context, what they should do if they are the recipient of inappropriate behaviour, what they should do if someone suggests their behaviour is inappropriate, and why it is important. Apart from the fact that most of us would prefer workplaces in which people are treated with respect, tolerating inappropriate behaviour leads to lost productivity, attrition of good staff and students, and, if allowed to escalate, to the loss of large amounts of time and energy in pursuing formal complaints procedures.

Finally, do professional bodies have a role in promoting appropriate behaviour and appropriate responses to inappropriate behaviour? Following two high profile cases of data fabrication in 2002, the American Physical Society surveyed (Physics Today, November 2004) its members to find out the state of ethics education and how ethical issues were addressed in practice within physics. A clear majority of members who had held a Ph.D. for less than three years felt that American Physical Society ethics statements should be broadened to include treatment of subordinates. In response the American Physical Society added a statement to its Code of Ethics that subordinates should be treated with respect and with concern for their well−being. This includes the responsibility of supervisors to mentor students, postdoctoral researchers, and employees with respect to intellectual development, professional and ethical standards, and career guidance. Could other professional bodies do more to promote high standards of behaviour in the workplace?

The Big Picture

The obstacles that women face in pursuing a career in science can conveniently be divided into three groups:

1. Structural issues such as the long training period, the predominance of insecure posts at early career level and that the period when people establish themselves as independent scientists coincides with the period of family formation.
2. Pragmatic issues: availability of affordable, high-quality childcare, difficulty of finding a job in the same geographic region as a partner, problems with travelling to attend conferences.
3. Cultural: stereo-typing, unconscious bias, family expectations.

These are not independent factors. In fact they interweave and feedback on each other. Family expectations can exacerbate pragmatic issues. Unconscious bias can magnify structural obstacles.

Not only are these factors inextricably linked but individually small disadvantages accumulate leading to lower overall success rates for women. The combination of circumstances that lead individual women to leave science will be unique for each woman. This means that attempts to isolate a single cause for the scarcity of women in science, for example, bias in appointments procedures, are unlikely to be successful.

It is apparent that the situation is different in different sciences. Very few women do physics, engineering and computer science in the first place (for UK figures see the UKRC Statistics Guide 2010) whereas in the UK over half of people attaining a postgraduate qualification by research in biological science are women (HESA, Table 7, Qualifications obtained by level, gender and subject area). It seems likely that the low numbers of women physics, engineering and computer science are predominantly due to cultural factors operating within schools. However, the example of the biological sciences shows that recruiting at the student level while clearly a necessary condition for increasing the number of women is not a sufficient condition. Structural and pragmatic issues must also be addressed if women are to be retained in STEM.

More on Quantitative vs Qualitative Evidence

One perceived disadvantage of qualitative studies is that analyzing the data, for example, responses to interviews, is inevitably done within some particular  interpretative framework. In fact, quantitative data is also interpreted within some framework but it is less usual to state what it is.

For example, the Asset 2010 survey of academic staff in SET departments found that 17% of women compared with 14% of men had no provision for appraisals, 9% of women compared with 7% of men could have an appraisal on request, and 74% of women compared with 80% of men had appraisals as a matter of routine. These differences are statistically significant.

There are a number of ways of interpreting this:

1. There are departments which discriminate against women by providing men with appraisals but not women.
2. Women are not uniformly distributed across disciplines. Perhaps some disciplines are more likely to operate appraisal schemes than others.
3. Perhaps men in departments with routine appraisals were more likely to have been encouraged to complete the survey.
4. The appraisal process itself discriminates against women so departments with routine appraisals are less likely to attract and retain women.
5. If men are more likely to receive routine appraisals, it is they who are discriminated against since they have to spend time filling in appraisal forms while their female colleagues get on with their jobs. (Presumably the slightly over a quarter of academic staff who reported that they did not find their appraisal to be either useful or valuable might concur with this interpretation.)

My point is not that any of these interpretations is particularly likely. It is that the same quantitative data can be viewed in different ways depending on your perspective. Those who are keen to establish that women are hard done by might incline to interpretation 1. Those who believe that universities are meritocracies and hence fair to all might incline to interpretation 2 or interpretation 3. Those with a less than positive experience of appraisal might incline to interpretation 4 or interpretation 5.

What then should the response be to findings such as these? The standard scientific response of wanting to eliminate incorrect explanations in order to isolate a single best explanation has the disadvantage that it could mean that action to correct an unfair situation is delayed, possibly for years. The other problem with this response is that the holders of strong views, wherever they lie on the spectrum between 'an appraisal is a good thing and everyone should have one whether they want it or not' and 'appraisals are a bureaucratic waste of time foisted on us by HR', rarely base their beliefs on rational analysis of evidence. Consequently finding more evidence is unlikely to change their minds. The standard top-down response of asserting one interpretation to be correct and labelling anyone who disagrees as 'obstructionist' or 'a dinosaur' has the disadvantage of being ineffective. Academics either ignore top-down initiatives or find ways of getting around them. My own view is that, at around a quarter, the proportion of academic staff finding their appraisals to be neither useful nor valuable is unacceptable high. The course of action I would favour is:

•  Explicitly recognise that people have different beliefs and experiences. Expecting someone whose experience of appraisal has been negative to be enthusiastic about a new appraisal policy is counter-productive.
•  Articulate what  a departmental staff appraisal policy is supposed to achieve
•  Ask staff for suggestions for how best to achieve it
•  Give staff feedback on their suggestions. (Too often, people make suggestions only for them to apparently disappear. For example, a suggestion may be perceived as being too resource intensive to implement. If the person making the suggestion is given this feedback they may well be able to think of ways of achieving the same result more efficiently.)
•  Formulate a policy that is grounded in reality and recognises the constraints on people's time. (In an ideal world everyone would attend training courses in how to appraise/be appraised. In real life they do not, unless they are compelled to do so, in which case they turn up, resent being there and don't learn anything.)
•  Monitor your procedures, not just by ticking off whether everyone has completed an appraisal but by seeking feedback on whether the procedures are achieving the desired results.

Disclaimer: My own experience of appraisal has been positive, though personally I would not rate appraisal as having been particularly useful to my career development, which may have more to do with my moves between New Zealand and the UK than the process itself.

When does evidence become evidence?

There are four broad categories of evidence that can be used to inform actions to improve the position of women in science.

The first category is institutional and national statistics, for example, what proportion of undergraduates, graduate students, post-docs, staff by grade are women? Such statistics are essential. You cannot even identify that you have a problem without them, or, indeed, that you have not. There are a number of difficulties with these data:

• The categories that were used to present the data may not be helpful. For example, physics and chemistry are often combined as 'physical sciences' although the participation rate of women in physics is much lower than that in chemistry.
• Aggregating data may obscure issues specific to a particular area or unit but reporting data at too fine a level makes it difficult to distinguish effect from random fluctuation. For example, I would be surprised if many individual departments (units of 20-50 academics) could make meaningful comparisons between the rate at which men are promoted and the rate at which women are promoted. Observations made with a sample size of three are effectively anecdotes, even if they are converted to a percentage and plotted on a graph.
• Snapshot data can be difficult to interpret. As Gillian Gehring put it in a comment on the interaction between gender, level of qualification and pay in Physics World in September 2001: “We need to think like an astronomer here: the women of 50+ graduated from an “earlier universe”.” (If you are not an astronomer the reference is to the fact that, when we observe an object in the universe now, the further away it is the longer ago that the light we now see was emitted.) Women who are now 50+  were 20+ thirty years ago. Things were different then.
• Even if you can demonstrate that there is a problem, the statistics by themselves throw no light on what has caused it. One approach is to try varying some practice while hoping that other factors remain the same and monitoring the statistics to see if they change. The disadvantages of this approach are that other factors do not normally remain the same, it can be difficult to ensure a uniform change in practice, especially in universities, and it can take an unfeasibly long time to have any confidence that you are observing a genuine change rather than a random fluctuation.

The second category is surveys. Surveys can be a very useful way of assessing how many women are affected by a particular issue. With the advent of tools like Survey Monkey they are technically very easy to set up though it still takes some effort to write good questions. Results can often be presented in numerical form and analyzed with conventional statistical tools, which is an advantage for those who are uncomfortable with qualitative data. The disadvantages are:

• Sample sizes may be small making it difficult to achieve statistical significance.
• It is difficult to assess how representative the sample is of the overall population. Respondents to surveys are often atypical at least in the respect that they have bothered to complete it.
• You only get responses to the questions that you thought of when you designed the survey. If someone raises a new issue in a free text response you have no way of knowing how many others might have agreed, unless you run a follow-up survey.

The third category is existing research. Usually you don't start a scientific project from scratch. You review the literature to see what is already known. Adopting a similar approach to tackling issues for women in STEM would be both more efficient and more effective than continually starting from scratch. Furthermore, some issues can only be identified via research projects. For example, the evidence for unconscious bias largely comes from published research studies. Research projects involving multi-variate analysis of reams of data can offer useful information about what issues affect the recruitment and retention of women but require considerable time and resources. The principal disadvantage with trying to find out what is already known is lack of time. Also, much experience is recorded in non-peer-reviewed reports published by organizations, for example the reports produced by the Royal Society of Chemistry, and can be hard to discover. We need more books like Virginia Valian's 'Why so Slow? The advancement of women' that pull the research together and present it in an accessible way.

Finally, there are qualitative methods such as interviews and focus groups. Natural scientists tend to be uncomfortable with qualitative data dismissing it as anecdotal. This is not surprising. You can't run a focus group for electrons to gather their experiences of being accelerated by an electromagnetic field or interview proteins about how they fold (though think how many person-hours of effort would be saved if you could). However, women are intelligent, articulate human beings. You can save a lot of time and effort by simply asking them what is important to them. All too often women's lived experiences are dismissed as irrelevant. There is an unfortunate connotation to ignoring qualitative data. Doing so sends the message: ‘Women are not capable of understanding or articulating their concerns. We need our armoury of experimental tools and statistical analysis to determine what is best for them.’

If the purpose of gathering the data is to identify the factors that affect women's progress in a particular institution and find ways of ameliorating their effects we do not need to apply the same criteria of rejection/provisional acceptance that we would apply if we were seeking a general predictive theory of why women are not thriving as well as might have been expected within the institution, assuming such a theory to exist, which I doubt. Decisions have to be made using the best information available now, not put off until sufficient data have been accumulated for a result to be considered statistically significant at some conventional level of significance. That does not mean we should ignore statistical significance. Devoting resources to fixing a random fluctuation is a waste of time and effort, though, of course, likely to appear to be successful.

Quantitative and qualitative methods are complementary. Quantitative methods are good for demonstrating that there is a problem: qualitative methods are good for generating insights into what might be causing the problem. Decisions should be made on the basis of all the available data.

Unconscious Bias

This is another long post. The essential points are:

• There is a body of evidence that women in STEM are adversely affected by unconscious bias.
• These implicit assumptions about women's roles (gender schemas) are a cognitive necessity for dealing with the social world but they can lead to inaccurate judgements.
• Men and women share these implicit assumptions.
• There are effective interventions that mitigate the effects of unconscious bias.

Way back in August I mentioned that there were two big ideas that made an impact on me when I started working for the Women in Science, Engineering and Technology Initiative at the University of Cambridge. The one I discussed in August was Teresa Rees's description of different approaches to gender equality as 'Tinkering, Tailoring and Transforming' – basically these correspond to removing discrimination, adapting women to fit the system and adapting the system to enable everyone to contribute to their full potential. The other big idea is the concept of unconscious bias.

I’ll start with a story. A few years ago when I was travelling on a train I glanced across the carriage and noticed that one of the passengers was reading ‘An Introduction to Electronics’.  ‘Ah’, I thought to myself,’ a young man using his commuting time to catch up on some study.’ A bit later I took a better look around the carriage and realised that the reader of ‘An Introduction to Electronics’ was, in fact, a young woman. I had spent around twenty-five years doing physics. Part of my job at the University of Cambridge was to run workshops on unconscious bias and women in science. To add to the irony the reason I was on the train was that I was returning from a meeting about a mentoring project for female undergraduate engineers organised by a female professor of electrical engineering. And my initial assumption was that someone reading a book about electronics would be male.

Unconscious bias refers to implicit expectations we all have about the roles and behaviour of members of particular groups. The reason this idea had a great impact on me was that it made sense of my experience that  people who professed a belief in equality, indeed were genuinely committed to achieving equality, nevertheless acquiesced in practices that unintentionally made it more difficult for women to progress in science and engineering.

I learnt about unconscious bias affecting women in professional careers from Virginia Valian’s book Why so Slow? The Advancement of women. Valian uses the phrase ‘gender schemas’ to describe the implicit hypotheses that we hold about sex differences. The content of these gender schemas does not depend on gender: men and women have the same beliefs. For example, a study of the relationship between the strength of an implicit association between ‘male’ and ‘science’ and a measure of achievement in science found that men and women had equally strong associations of ‘science’ with ‘male’ [Nosek et al, 'National differences in gender-science stereotypes predict national sex differences in science and math achievement'. Proceedings of the National Academy of Sciences, 106, 10593-10597 (2009) ]. Valian emphasizes that gender schemas are a cognitive necessity for making sense of the social world. However, in some contexts, they can lead to lower expectations for women.

Subsequently there have been two frequently cited studies reinforcing the points made by Valian. One, Steinpreis et al, 'The Impact of Gender on the Review of the Curricula Vitae of Job Applicants and Tenure Candidates: A National Empirical Study' Sex Roles 41, 718 (1999) (pdf), reports a study in which CVs that differed only in whether the name was given as Karen Miller or as Brian Miller. The study found that both men and women were more likely to vote to hire a male job applicant than a female job applicant with an identical record. In the other study Trix and Psenka, 'Exploring the Color of Glass: Letters of Recommendation for Female and Male Medical Faculty' Discourse and Society 14, 191 (2003) found that letters of recommendation for women were shorter, contained twice as many ‘doubt raisers’ (‘she has a somewhat challenging personality’), more ‘grindstone adjectives’ (conscientious, diligent) and fewer ‘stand-out adjectives (superb, outstanding). ( Similar results have been obtained by Schmader et al in a study of letters of recommendation for faculty positions in biochemistry and chemistry,' A Linguistic Comparison of Letters of Recommendation for Male and Female Chemistry and Biochemistry Job Applicants', Sex Roles 57, 509 (2007).)

The Project Implicit website has on-line tests that measure the strength of implicit associations such as that between ‘male’ and ‘science’.

What can we do about it?

The Project Implicit FAQ suggests that individuals who find that they have an implicit bias that they would rather not have could try seeking experiences that contradict their implicit bias, being conscious of their bias and its potential effect on their judgements and consciously planning actions that compensate for their known unconscious preference.

Valian suggests:

• Learning about gender schemas, accepting that we may have expectations that we are unaware of that may contradict what we consciously believe and consciously changing our behaviour.
• Challenging implicit hypotheses, for example, by imagining our response to someone’s behaviour if the person concerned was male rather than female.
• Reducing reliance on gender schemas by
• Spending more time on decisions. People use gender schemas to make automatic responses. If more time is available there is less need to rely on an automatic response.
• Giving decisions our full attention. Making judgements while distracted by another task increases the reliance on gender schemas.
• Holding decision makers accountable. People are more likely to form accurate judgements if they know there decisions will be reviewed.
• Increasing the number of women in the candidate pool. Gender schemas become less important when women form a reasonable proportion of the group being evaluated.
• Avoiding errors of reasoning such as:
• Failure to appreciate covariation, where for example, an apparent variation with sex in some ability is actually due to some other variable.
• Blocking, where the fact that data are consistent with a prevailing schema prevents evaluators from noticing other factors that have affected performance.
• Illusory correlation, where people perceive a causal link between rare events such as incompetence and being female, simply because both events are rare.
• 
  
• At an institutional level
• Committed leaders and leaders who are ready to legitimize woman leaders.
• Objective performance criteria.
• 
  
• For women – how can they increase the chances of being perceived as competent?
• Be where women are well represented, which isn’t a lot of help to physicists, mathematicians, engineers and computer scientists.
• Be impersonal, friendly and respectful. This minimizes the negative effects of being perceived as competent.
• Build power
• Seek information.
• Become an expert.
• Get endorsed by a legitimate authority.
• Negotiate
• Overcome internal barriers. One of the effects of gender schemas is that women may attribute lack of reward of their efforts to their lack of ability. A better understanding of gender schemas and how they work.
• (For practical suggestions on how to accomplish the above, see  Suzanne Doyle-Morris’s book, Beyond the Boys’ Club.)

The STRIDE programme at the University of Michigan is an example of a comprehensive, successful intervention.

Useful resources:
1. Virginia Valian’s ‘Tutorials for Change'

2. The University of Michigan’s STRIDE Faculty Recruitment Presentation, available from the STRIDE website

3. The report of the US National Academies 'Beyond Bias and Barriers: Fulfilling the potential of women in science and engineering' National Academies Press (2007).

4. WISELI (Women in Science and Engineering Leadership Institute), University of Wisconsin-Madison Reports and Publications – A particularly intriguing report listed here is ‘Interventions That Affect Gender Bias in Hiring: A Systematic Review’ by Isaac, Lee and Carnes  Academic Medicine 84, 1440 (2009).This report analysed 27 previous studies of interventions that affect gender differences in the evaluation of job applicants. The authors found that the studies showed a negative bias in evaluations of women for positions in areas traditionally or predominantly held by men and also that the assessments by men and women rarely differed. Interventions that were effective in mitigating the effects of unconscious bias were:
-    Providing assessors with clear evidence of job-related competencies, with the proviso that additional evidence of ‘communal qualities’ was provided for women.
-    A commitment to the value of credentials before reviewing the applicants.
-    Women’s presence at greater than 25% of the applicant pool.
Two studies found unconscious resistance to anti-bias training. And, according to the abstract:

‘Explicit employment equity policies and an attractive appearance benefited men more than women, whereas repeated employment gaps were more detrimental to men. Masculine-scented perfume favoured the hiring of both sexes.’ (My emphasis).

Videos to catch

In the absence of anything to watch on TV over the holidays, I've been watching videos from the internet. Here are three that I've found particularly interesting:

'Why we have too few women leaders' by Sheryl Sandberg, Chief Operating Officer at Facebook (approx 15 min). While she acknowledges that changes in business practices are necessary, for example, flexible hours, in this talk she focuses on what individuals can do. Her three points are:

• ‘Sit at the table’ – Women tend to underestimate their own abilities, they don't negotiate higher salaries and while men attribute success their abilities women tend cite external factors. One of the issues is that while successful men are perceived as likeable successful women are not.
• ‘Make your partner a real partner’ – sharing the chores is good for your marriage.
• ‘Don't leave before you leave’ – Don’t turn down that promotion or that new project because you are planning on having a child, especially not if you don’t even currently have a boyfriend.

Women need to watch this video. So do managers.
 
Margaret Wertheim on the beautiful math of coral in which Margaret Wertheim reveals the link between crochet, coral and hyperbolic geometry (approx 15min). She makes an interesting point about our tendency to value symbolic knowledge, marks on paper or on a computer screen, over embodied knowledge, things we can touch and feel.
 
The truth about denial by Naomi Oreskes, Professor of History and Science Studies at the University of California San Diego. This is a video of a talk given in 2007. The first part covers the history of  the science of global warming from Tyndall to the Fourth Assessment Report of the IPCC. The second part addresses the question, why, if the science is uncontroversial, do so many Americans think it is still doubtful? In 1992  US President George H. W. Bush signed the United Nations Framework Convention on Climate Change. Why in 2007 were we still arguing about whether global warming was even happening? (approx 1 hour)

Simpson's Paradox

Suppose there are 300 students doing a Maths exam, 180 men and 120 women, and another 200 students doing an English exam, 60 men and 140 women. Suppose 30% of the Maths students and 20% of the English students get an A. If the performance of male and female students is exactly the same then the results will be as shown in the Table:

	Proportion of men getting an A	Proportion of women getting an A	Overall Proportion getting an A
Maths	54/180 = 0.3	36/120 = 0.3	90/300 = 0.3
English	12/60 = 0.2	28/140 = 0.2	40/200 = 0.2
Combined	66/240  = .2750	64/260 = 0.2462	130/500 = 0.2600

So, despite the fact that there is no difference in men's and women's performance on the individual exams, overall 27.5% of men got an A and only 24.62% of women got an A. This phenomenon in which the trend in amalgamated data is different from the trends in the individual groups is known as Simpson's Paradox. A famous example is graduate admissions at UC Berkeley in 1973: 44% of the 8442 male applicants were admitted compared with 35% of the 4321 female applicants, a difference that was too large to be due to chance. However, when the data were disaggregated by decision making unit a different picture emerged: few units showed a statistically significant difference between  the rates of male and female admissions and there were just as many units that appeared to favour women as to favour men. The reason for the discrepancy in the overall admission rates was that women were proportionately more likely to apply to units with low admission rates. In fact, when the data were pooled taking this into account there was a small but statistically significant bias in favour of women. [P.J.Bickel, E.A. Hammel and J.W.O'Connor Science, vol 187 pp 398-404 (1975)].

The Berkeley example dates from over thirty-five years ago. Do we still see this type of error in the analysis of gender statistics? Yes, we do.

I speculate that the reasons for this include:

• A preference for results that confirm our existing beliefs. If we believe that women are hard done by we are less likely to question results that support that belief.
• A fear that questioning specific results will be perceived as questioning the principle of equality or will be interpreted as a reason for failing to support an otherwise useful initiative.
• Lack of knowledge of statistics and its pitfalls.

Additional Sources: I first came across the Berkeley example in a video of a lecture 'Lies, Damned Lies and Statistics: The misapplication of statistics in everyday life' by Dr Talithia D. Williams,  a lecture in the Distinctive Voices series of the US National Academy of Sciences. It's well worth watching.

Equal Pay

This is, once again, a somewhat technical post. It highlights some of the shortcomings of focussing on gender pay gaps as a means of identifying biases in pay. It also raises the question: what is a practically important gap? I do not have a good answer to that question. It would be nice to see some discussion of what makes a pay gap important for practical purposes. A related question is: how much effort should employers put into addressing a pay gap that is in some sense ‘large’ but that has a high probability of being due to chance (i.e. is not statistically significant), bearing in mind that resources that are used for one activity are not available for other activities? Again, there is probably not a unique answer to this question but it would be nice to see it discussed.

The principle of equal pay is equal pay for:

•    Equal work – work that is the same or broadly similar.
•    Equivalent work – work that has been rated as equivalent by a job evaluation scheme.
•    Work of equal value – work that places similar demands on those performing it.

As noted by the Equality and Human Rights Commission (EHRC) the overall gender pay gap is not an indicator of unequal pay. They suggest that it should be seen as an ‘equal opportunity gap’. The analysis in my earlier post suggests that it is not a particularly useful measure of that either. Nevertheless, it is popular.

The tool to check for equal pay is an equal pay audit. Equal pay audits should be carried out for ethnicity and disability as well as gender but gender is often easier since the quality of the data is better. Basically the steps in an equal pay audit are:
1. Determine which people are doing equal work, equivalent work or work of equal value. An organisation that has a job evaluation scheme may assume that everyone with a job evaluated to be in the same grade is doing equivalent work or they may split people on the same grade into groups doing similar work. The latter procedure is preferable if there are enough employees since aggregating over all the employees in a grade could mask problems that were specific to one group.

2. Assess whether men and women are equally paid. There are two situations that might apply: there is systematic bias against women (or men) or a few individual women (or men) are disadvantaged, for example, the way in which starting salary is determined may disadvantage people returning from a career break which tends to disadvantage some, but not all, women. Although carrying out an equal pay audit may uncover instances of the latter it is more usual to concentrate on the former, though both are illegal. The EHRC suggests that the first step is to calculate the average basic pay for men and women and the average total pay for men and women. The EHRC recommends that if the difference between the average pay for men and the average pay for women is greater than 5% or if the difference between the average pay for men and the average pay for women is greater than 3% and there is a pattern of gaps favouring one sex over the other then further investigation is required. 

Most scientists asked to determine whether the difference of two means indicated the presence of systematic bias would probably start by trying a t-test. There are some problems with this approach, especially if the means are calculated from a small number of employees, because t-tests are based on the assumption that the estimate of the mean has a normal distribution. Tests of statistical significance, calculate the probability of observing a difference of at least the size you did observe on the assumption that there is no difference. The larger the value of this probability the more likely it is that the observed effect could have arisen just by chance even if, in fact, there is no difference.  It is not necessary to adopt the convention that a result is significant if the probability of getting a result at least that big is less than 0.05 and not otherwise. Depending on the cost it might be reasonable to take action even if the probability that the observed difference is due to chance is 0.3, or even more, depending on the circumstances. Note that observing an effect that is not statistically significant does not imply that there is no effect. It implies that there is not enough data to say whether or not there is an effect.

As noted by the EHRC, statistical significance should not be confused with effect size  (see Technical Note 3.5). A large difference can fail to be statistically significant if there are a small number of employees, or a small number of employees of one sex. A small difference can be statistically significant if there are a large number of employees. So, for example, an institution might be more worried about a 15% gender pay gap that had a 10% probability of being exceeded due to chance than about a 1% gender pay gap that had a 4% probability of being exceeded due to chance. In the latter case the institution is saying that they are willing to accept a low probability that the observed result is due to chance since they believe the bias to be too small to be of practical importance. What constitutes a large difference? What is practical importance?

The EHRC criteria are that if, on average, one sex earns more than 5% more than the other for doing particular equivalent jobs, or more than 3% more if there is a pattern, then there is a problem that needs investigating. These criteria could lead to anomalies depending on how pay is determined. Organizations that rely on negotiation by individuals to set pay or on pay schemes with a significant component dependent on performance evaluation are open to inadvertent discrimination leading to systematic discrepancies that would be evident in gender pay gaps. Other organizations use a system in which the person identifying the need for a position writes a job specification that is used by a professional job evaluator to assign a grade to the job with the person appointed to the position being assigned to a point within the grade on the basis of their qualifications and experience and then progressing by automatic annual increments to a point at which he or she needs to apply for promotion to discretionary or contribution points of the grade. Under this system there is much less scope for discrimination. Possible ways in which bias can occur are:
1.    There could be a tendency for women to be appointed at a lower point in the grade.
2.    Women could be less likely to apply for promotion to discretionary or contribution points.
3.    The job evaluation scheme could result in jobs predominantly done by men being graded higher than jobs predominantly done by women.
4.    Women could be less likely to receive, or receive lower amounts of, additional payments such as allowances, payment for additional responsibilities, recruitment incentives or market supplements.
5.    There could be differences in the contractual hours of different occupational groups in jobs evaluated to be at the same grade.
6.    There could be differences in pension entitlements or retirement ages between different groups with different representations of men and women.

Two factors which could influence gender pay gaps but which are not equal pay issues are:
1. Women might be more likely to leave giving a greater proportion of women on lower points in the grade or men might be more likely to leave, for example, for higher graded positions, leaving proportionately more women at the top of the grade.
2. Women may have been entering jobs at this level in increasing numbers in recent years leading to a clustering of women at lower points in the grade.

We thus have factors which are equal pay related that will be reflected in gender pay gaps, such as lower starting salaries, factors which are equal pay related that will not be reflected in equal pay gaps, such as biases in the job evaluation scheme, and factors that are not equal pay related but will affect the gender pay gap.

Figure 1 shows a distribution of women on a nine point scale, perhaps arising as a combination of women ending up with lower starting salaries, being less likely to apply for promotion and men leaving for better paid positions. The trend line has a slope of -0.0425, so the proportion of women falls by about 4 percentage points per scale point. If there are the same number of people on each salary point and each salary point is 2.5% higher than the one below then this distribution leads to a gender pay gap of 2.8% and no investigation is required. If, however, each salary point is 5% higher than the one below the gender pay gap is 5.7% and further investigation is required although the underlying biases that led to this situation would be the same in both cases. Both gaps are statistically significant if there are twenty or more people on each salary point.

Figure 2 shows a similar distribution on a fourteen point scale. The trend line has a slope of -0.0312. In this case the gender pay gap is 5.1% when the increment from one scale point to the next is 2.5%. This gap is highly statistically significant if there are at least twenty people on each scale point. Note that if this grade was split into two grades of seven scale points the gender pay gaps would be 1.6% for each grade though the men and women would be being paid the same salaries as before.

These examples show that the same underlying biases can give rise to gaps that may or may not be regarded as practically important depending on the particular salary structure.

As another example, suppose you have 100 men doing a particular job at a particular grade with an average salary of £25,000 and 100 women doing the same job at the same grade who would have the same average salary except that a policy of taking existing salary into account when determining the starting point in the grade has led to twenty women who returned from a career break being paid 6% less than similarly qualified men or women who had not taken a career break. The average salary for all the women is £24717, a gap of 1.2%. This is not likely to be statistically significant (on a ten point scale with an average of twenty people per scale point with a 3% increment there is a 15% chance of men’s average pay exceeding that of women by at least 1.2%) and nor is it large, though the twenty affected women might disagree.

This example shows that relying on the gender pay gap to identify anomalies could result in failing to detect substantial biases.

This still leaves the question: what is a practically important gap? There does not seem to be a good answer to this question. Is it acceptable for women to paid one scale point less than comparable men as long as the gap between scale points is less than 3% but not acceptable if the gap between scale points is greater than 3%? Is exactly the same bias acceptable if it occurs over two grades with a small number of steps but not if it occurs over one grade with a larger number of steps? Is it acceptable for 50% of women to be paid one scale point less than comparable men but not for 100% of women? What are the criteria for practical importance? Practically important to whom? The women earning 6% less than they might have been? The employer who might face an equal pay claim?

Does this mean carrying out an equal pay audit is a waste of time? No, it does not. An organization that measures its gender pay gaps for groups identified as doing the same or equivalent work or doing work of equal value is more likely to identify anomalies than one that does not. What it does mean is that organizations should examine their pay schemes and identify how anomalies could occur, for example, that people returning from career breaks tended to be placed on lower starting salaries thus tending to disadvantage women, and monitor those points directly regardless of whether or not they have observed substantial gaps.

An effective equal pay audit would:
•    Describe the way the institution sets pay or, at the very least, refer to another document that does so.
•    Identify the processes where bias could occur, e.g. setting starting salaries.
•    Monitor those processes.

 It could be that in a system where individuals negotiate their own pay or individuals’ line managers have a large say in setting performance pay that the best way of monitoring is to measure the gender pay gap. In institutions with job evaluation schemes and set pay scales it would be better to monitor starting salaries and progression directly to avoid potential bias being masked by other factors that affect the pay gap.

Processes that could introduce bias include the job evaluation scheme, if one is in place. The EHRC has pertinent advice on how to check that a job evaluation scheme does not itself inadvertently discriminate against women.

Has your institution carried out an equal pay audit? Does it meet the above criteria? Do you think it should? Is there anything you can do about it if it doesn’t?