Skip to content

Oh, behave: Predictive, convergent and discriminant validation of the qIAT

ינואר 1, 2013

Abstract

The purpose of the following study was to establish the qIAT as a valid and reliable tool for the indirect administration of existing, and future, psychological assessment and evaluation questionnaires. Administrating psychological assessment questionnaires in an indirect manner could circumevent some biases and external influences assosciated with using explicit measures. Personality assessment and measurement, as the uprooting domain of the qIAT, was naturally chosen as the ground to demonstrate its psychometric qualities.

Adopting the theoretical framework presented by Back et al. (2009; i.e., the BPMP), the study applied Yovel and Friedman's (2012) qIAT on the Big-Five dimensions of Agreeableness, Conscientiousness, and Neuroticism, and explored the qIAT's predictive validity of behavior, using behavioral criteria developed in previous theoretical as well as experimental research. It was hypothesized that the implicit measure of personality assessment, the qIAT, would correlate significantly with the explicit measure of personality assessment (i.e., IPIP questionnaire) on the corresponding dimension (i.e., convergent validity), but that it would not correlate highly on all other dimensions (i.e., discriminant validity). In addition, it was hypothesized that the implicit measure could predict behavior on the corresponding dimension and that the explicit measure of personality could also predict behavior on the corresponding dimension.

Outliers cases were winsorized, and the convergent and discriminant validities of the qIAT were supported. However, predictive validity of the qIAT was not significant in the majority (92%) of the behavioral criteria defined a priori. In addition, predictive validity for the explicit measure of personality was also non-significant for the majority (89%) of defined behavioral criteria.

Some findings of the present report stand in disaccord with existing research literature, and possible explanations for them were discussed, as well as limitations of the current study. Future research directions were suggested, mainly a replication of the current design with a larger sample, and applying the qIAT as an indirect administration of existing psychological assessment and evaluation questionnaires in fields other than personality assessment and behavior prediction.

You are what you do, not what you say you'll do.”   -Carl Gustav Jung

Introduction

Predicting behavior seems to be one of psychology's most ambitious and time-lasting goals. Ever since Charcot and Freud's attempts at modeling a set of symptoms as derived from inner-psychic factors (Freud, 1917), psychologists have been trying to understand and model the relationship between inner and outer reality. Of course, this effort has some substantial incentives – being able to evaluate and predict job performance (Feuerhahn, Kühnel, & Kudielka, 2012), psychologically profiling an offender in a crime investigation (Schlesinger, 2009), or predicting suicidal attempts (Nock et al., 2010) could prove very valuable to society.

In this endeavor, a fruitful concept has been personality, the ways in which individuals differ in their enduring emotional, interpersonal, experiential, attitudinal, and motivational styles. A common personality taxonomy is the five-factor model (McCrae & John, 1992), which assumes that five broad and robust factors, often referred to as the Big Five, account for a considerable amount of covariation between personality traits. These factors, or dimensions, are commonly known as Openness to Experience (Intellect), Conscientiousness, Extraversion, Agreeableness and Neuroticism. (McCrae & Costa, 1987; McCrae & John, 1992).

Based on Caspi, Roberts and Shiner's (2005) attempt to elaborate and expand the Big Five's definition by listing the lower-order, more specific, traits of each dimension, a brief synopsis will be made of the different prototypical individuals, high-trait and low-trait, of each factor:

Neuroticism's lower-order traits are inner-focused such as a tendency toward anxiety, sadness, insecurity, and guilt, as well as outer-directed such as hostility, anger, jealousy, frustration, and irritation. Neurotic individuals are anxious, vulnerable to stress, guilt-prone, lacking in confidence, moody, angry, easily frustrated, and insecure in relationships; individuals low on this trait are emotionally stable and adaptable.

Conscientiousness' lower-order traits are self-control, attention, achievement motivation, orderliness, responsibility, and conventionality. Conscientious individuals are responsible, attentive, careful, persistent, orderly, and tend to make plans and act upon them; those low on this trait are irresponsible, unreliable, careless, and distractible.

Agreeableness's lower-order trait include prosocial behaviors, such as a tendency to be helpful, kind, considerate, generous, empathic, and nurturing, as well as cynicism/alienation and antisocial tendencies such as physical and relational aggression. Agreeable individuals are cooperative, considerate, empathic, generous, polite, and kind. Disagreeable individuals are aggressive, rude, spiteful, stubborn, cynical, and manipulative.

Extraversion's lower-order trait are social inhibition or shyness, sociability, dominance, and energy/activity level. Extraverted individuals are outgoing, expressive, energetic, and dominant, whereas introverted individuals are quiet, inhibited, lethargic, and more content to follow others’ lead.

Openness' lower-order traits are imaginative ability, creativity, and aesthetic sensitivity, as well as quick learning, cleverness and being insightful. An individual with high Openness is artistic, curious, imaginative, insightful, creative, and has wide interests; an individual with low Openness is shallow, unadventurous, imperceptive, and has narrow interests (Caspi et al., 2005; McCrae & Costa, 1987; Shiner, 1998). Two cardinal points are that Openness is measured less reliably than the other Big Five traits (see, for example: Shiner, 1998), and that it is the most debated, least defined, and least understood of the Big Five traits (Raad, 2006).

Shiner's (1998) and Raad's (2006) claims about Openness, and some of the more inner-focused lower-order traits, and therefore less accessible to direct observation (e.g., feelings of guilt), raise a bigger issue: how does one measure personality? A common, and quite compelling, maxim is self-report. However, self-reports are vulnerable to implicit response tendencies that respondents might not be aware of (Greenwald & Banaji, 1995; Wilson, 2009), as well as social desirability effects, impression management and self-deception strategies (Greenwald & Banaji, 1995; Paulhus, 1984; cf. Rohner & Björklund, 2006). In paraphrase on C.G. Jung's quote, people are what they do, not what they say they do – asking them what they are simply isn't the right way to go.

Where explicit measures fail, implicit measures prosper. Recent years have brought an amplitude of tools and research paradigms that intent tapping the implicit aspects of people's inner-reality (Bosson, Swann, & Pennebaker, 2000). One prominent paradigm, the Implicit Association Test (IAT), was developed by Greenwald, McGhee, and Schwartz (1998). The IAT is a double category-discrimination task that can measure participants' strength of association between two concepts by using two sets of antonyms (e.g., Good versus Bad and Flowers versus Insects). The IAT has been used to asses and predict behaviors related to implicit attitudes, race-related stereotypes (Greenwald et al., 1998), self-esteem (Greenwald & Farnham, 2000), anxiety levels (Egloff & Schmukle, 2002), cognitive abnormalities such as pedophilia (Gray, Brown, MacCulloch, Smith, & Snowden, 2005) and personality (Back, Schmukle, & Egloff, 2009; Schmukle, Back, & Egloff, 2008).

An additional step in the evolvement of IAT was the Autobiographical IAT (aIAT), presented by Sartori, Agosta, Zogmaister, Ferrara, and Castiello (2008). While all aforementioned IAT paradigms used single words as stimuli, Sartori and colleagues' modified paradigm used more complex stimuli, complete sentences (Sartori et al., 2008). In addition, the aIAT, developed as a lie-detection tool and designed to be used in forensic settings, used as its sets of antonyms True–False and Guilty–Innocent.

Further development of the IAT paradigm made recently by Yovel and Friedman (2012) is the questionnaire-based IAT (qIAT), a task designed to enable indirect measurement of standard self-report questionnaires. This task employed the aIAT's use of complete-sentence stimuli (i.e., complete items from explicit personality questionnaires) and the True-False antonym, but used Extraversion, a domain of the Big-Five, as the scale of the second antonym set (i.e., Extrovert–Introvert).

As implied earlier, implicit and explicit measures of personality assessment are not expected to correlate perfectly. Other than methodological issues that might impede each measures' validity (e.g., Social Desirability in explicit measures (Paulhus, 1984), or base rate effects in implicit measures (Bluemke & Fiedler, 2009)), it is possible that implicit and explicit personality measures simply tap different, yet sometimes not unrelated, psychological constructs (Hofmann, Gawronski, Gschwendner, Le, & Schmitt, 2005). Recent literature suggests new models to explain and predict actual behavior, by accounting for both the implicit and explicit aspects of people's personality and modes of thinking (see, for example: Dijksterhuis & Nordgren, 2006). Some of these models, such as Back, Schmukle, And Egloff's Behavioral Process Model of Personality (BPMP; 2009), incorporate personality into the mix.

The BPMP suggests that actual behavior is the result of behavior schema being activated beyond a certain threshold, an activation that can be caused by either impulsive or controlled processes. The model conceptualizes personality as the typical functioning (across time and multiple situations) of both kinds of processes, and therefore as having the potential of being twofold: a person has one reflective process personality (i.e., an explicit self-concept) and one impulsive process personality (i.e., an implicit self-concept). While explicit measures of personality tap the reflective processes – how people typically perceive and categorize situations, which behavioral options they prefer, and how they deliberately realize these preferences – implicit measures tap impulsive processes – how situational cues are automatically processed and what kinds of actions are automatically performed. According to the model, in order to predict actual behavior, both the explicit and the implicit self-concepts of personality (i.e., the typical functioning of both reflective and impulsive processes) are to be taken into consideration (Back et al., 2009).

Even when using the Big-Five taxonomy and taking into account both the implicit and explicit aspects of personality, upon trying to predict actual behavior in an experimental setting the question remains, which behaviors correspond to each personality type? Previous work, such as that of McCrae and Costa (1987), McCrae and John (1992), Halverson et al. (2003), Caspi et al. (2005), and Back, Schmukle, and Egloff (2006) provide good theoretical background for compiling a diverse set of relevant social situations, including speaking situations, interpersonal interactions, creativity tasks, helping situations, performance tasks, and unstructured situations. In addition, experimental research, such as that of Grucza and Goldberg (2007), Back et al. (2009), Jackson et al. (2010), Thalmayer, Saucier and Eigenhuis (2011) and Yovel and Friedman (2012) provides an ample pool of tasks from which adaptations can be made. So as not to burden the reader, theoretical foundations for each behavior criterion used in the current study are provided in the Methods section, hereinafter.

The present study, while adopting the theoretical framework presented by Back et al. (2009; i.e., the BPMP), attempts to apply Yovel and Friedman's (2012) qIAT on the Big-Five dimensions of Agreeableness, Conscientiousness, and Neuroticism, and explore the qIAT's predictive validity of behavior, using behavioral criteria developed in previous theoretical as well as experimental research. It is hypothesized that the implicit measure of personality assessment, the qIAT, would correlate significantly with the explicit measure of personality assessment (i.e., IPIP questionnaire) on the corresponding dimension (i.e., convergent validity), but that it would not correlate highly on all other dimensions (i.e., discriminant validity). In addition, it is hypothesized that the implicit measure could predict behavior on the corresponding dimension. An additional hypothesis, inline with existing research literature, is that the explicit measure of personality could also predict behavior on the corresponding dimension.

The purpose of the following experiment is to establish the qIAT as a valid and reliable tool for the administration of existing, and future, psychological assessment and evaluation questionnaires. Administrating psychological assessment questionnaires in an indirect manner could circumevent some biases and external influences assosciated with using explicit measures. Personality assessment and measurement, as the uprooting domain of the qIAT, was naturally chosen as the ground to demonstrate its psychometric qualities. For that reason, the qIAT must show that it is valid and reliable as an indirect personality assessment measure.

Failure to demonstrate these validities would raise doubts as to the psychometric qualities of the qIAT and its ability to act as a tool for the administration of questionnaires. However, failure to demonstrate these effects could also mean that while personality assessment was its uprooting domain, the qIAT might benefit if tested in other, less ambigious, fields (e.g., assessing of anixety or suicidal tendencies).

Methods

Participants

Participants were 49 consented undergraduates (55.1% females; Mage = 24.92, SDage = 2.33), who received course credit or monetary compensation. All participants had normal or corrected-to-normal vision and were native speakers of Hebrew.

Personality measurement

Explicit measures

Standard self-report assessment included the 50-item International Personality Item Pool questionnaire (IPIP; Goldberg, 2005), which measures the personality domains of the Big-Five factor structure (Agreeableness, Conscientiousness, Extraversion, Emotional Stability, Intellect; (McCrae & Costa, 1987)). Each dimension was measured by a 10-item subscale. Items were rated on a 1–5 Likert scale. Internal consistencies (Cronbach’s Alphas) in the current study were .93 for Extraversion, .92 for Emotional Stability (Neuroticism), .91 for Conscientiousness, .89 for Agreeableness and .82 for Intellect (Openness).

Implicit measure

Implicit measurement was based on the qIAT (Yovel & Friedman, 2012), a brief classification task in which the general methodology of the aIAT (Sartori et al., 2008) was followed. Each participant completed 5 qIAT measurements, one for each domain of the Big-Five factor structure (Agreeableness, Conscientiousness, Extraversion, Emotional Stability, Intellect; McCrae & Costa, 1987). The qIAT included several blocks. On each trial a sentence was presented at the center of the computer monitor, and participants were asked to classify it as quickly and accurately as possible using one of two designated response keys. In Block 1 (40 trials), participants were introduced to the classification of the personality categories, labeled as a high-trait person (e.g., a high-trait person for Extraversion would be 'extravert') versus low-trait person (e.g., a low-trait person in Extraversion would be 'introvert'). Participants were also introduced to the corresponding lists of items from the IPIP (e.g., under 'extravert', five non-reversed extraversion items; under 'introvert', five reversed extraversion items; all stimuli are presented in Table 1). In Block 2 (20 trials), they were introduced to the classification of the self-related logical categories, labeled true (e.g., "I’m participating in an experiment in psychology") versus false (e.g., "I’m shopping at the local grocery store"). In Block 3 (20 trials) and Block 4 (40 trials), participants performed these tasks interchangeably (first double categorization; e.g., extravert person and true versus introvert person and false). In Block 5 (40 trials), they practiced the reversed classification of the personality category, and in Blocks 6 and 7 (second double categorization), they again classified the sentences based on both categories, this time using the reversed trait classification (e.g., introvert person and true versus extravert person and false). The original qIAT included the seven aforementioned blocks. However, in the present study, Block 2 (i.e., practice of true-false categorization) was omitted in all but the first qIAT task. In all trials, the labels of the categories remained on the computer screen as a reminder, and an error signal appeared after an incorrect response (i.e., erroneous classification). Personality items and true versus false sentences were presented in alteration in the double-categorization blocs. The order of the double-categorization blocks was counterbalanced across participants. Reaction-times and error responses for all trials were recorded. Participants viewed the display from a distance of 45 cm, which was controlled by a chin rest.

For each participant five D scores were calculated, one for each trait, following Greenwald, Nosek, and Banaji (2003) improved scoring algorithm. Larger positive D’s represent a stronger association between the non-reversed trait items and the self-related true category. Thus, similarly to the total score of the self-report instrument, individuals higher on a specific trait are expected to have larger D’s on that trait's qIAT measurement. Internal consistencies (Spearman-Brown) in the current study were .84 for Neuroticism, .81 for Extraversion, .81 for Openness, .76 for Agreeableness and .73 for Conscientiousness.

Table 1 Stimuli used in the qIAT

True category

False category

I’m in a building in Mount Scopus campus

I’m climbing a steep mountain

I’m in a small room with a computer

I’m sitting on the sand at the beach

I’m participating in an experiment in psychology

I’m playing my electric guitar

I’m in a psychology laboratory

I’m playing soccer outside

I’m sitting in front of the computer

I’m shopping at the local grocery store

High-trait person

Low-trait person

Extraversion

I am skilled in handling social situations. I find it difficult to approach others.
I feel at ease with people. I don't talk a lot.
I am the life of the party. I am quiet around strangers.
I feel comfortable around people. I keep in the background.
I start conversations. I often feel uncomfortable around others.

Agreeableness

I love to help others. I am not really interested in others.
I sympathize with others' feelings. I am indifferent to the feelings of others.
I feel others' emotions. I feel little concern for others.
I make people feel at ease. I insult people.
I am interested in people. I am not interested in other people's problems.

Conscientiousness

I like order. I make a mess of things.
I like to tidy up. I leave my belongings around.
I follow a schedule. I leave a mess in my room.
I am exacting in my work. I shirk my duties.
I do things according to a plan. I often forget to put things back in their proper place.

Neuroticism

I am relaxed most of the time. I get irritated easily.
I am not easily bothered by things. I am easily disturbed.
I seldom feel blue. I have frequent mood swings.
I rarely get irritated. I worry about things.
I seldom get mad. I panic easily.

Openness

I love to read challenging material. I do not have a good imagination.
I am full of ideas. I am not interested in abstract ideas.
I carry the conversation to a higher level. I have difficulty imagining things.
I am good at many things. I avoid difficult reading material.
I have excellent ideas. I have difficulty understanding abstract ideas.

Behavioral Observations

For each of the five personality dimensions, a number of behavioral observations were defined a priori. In predefining criteria, conceptual descriptions of the Big-Five Domains were used as reference  (Back et al., 2006; Caspi et al., 2005; Halverson et al., 2003; McCrae & Costa, 1987; McCrae & John, 1992) as well as prior research on behavioral personality correlates (Back et al., 2009; Grucza & Goldberg, 2007; Jackson et al., 2010; Thalmayer et al., 2011; Yovel & Friedman, 2012).

A number of behavioral criteria were objectively measured by counting behavioral occurrences (e.g., lateness of attendance). Previous research (Back et al., 2009) used a German adaptation of the Linguistic Inquiry and Word Count (LIWC; Pennebaker & Chung, 2007) to objectively measure linguistic properties of texts composed by participants. However, since there is no Hebrew LIWC adaptation available, said bodies of texts were individually rated by the author and a colleague (e.g., use of aversive-aggressive concepts or words in Self-Introduction). Inter-rater correlations were calculated and are reported where applicable.

Although all dimensions of the Big-Five were tested, only Agreeableness, Conscientiousness, and Neuroticism will be discussed in detail in the present study. Hence, aspects of tasks designed to measure Extraversion and Openness will be mentioned where relevant, though not fully explored.

Time of Arrival

Participants were required to enlist to the experiment at least 30 minutes prior to the beginning of the session. As was done in previous research (Back et al., 2006, 2009), upon participant's arrival, the experimenter noted the time. Difference between appointed and actual time was calculated in minutes and recorded.

Story

Based on a task used by Back et al. (2009) Participants were given 7 minutes to write a short story in which the words air crash, firework, chambermaid, middle-ages, and supermarket were included. Failure to include or misspelling one of the required words was coded as an error, for the list of words remained visible throughout the 7 minutes. However, since, unlike English, Hebrew is a highly morphological language, all grammatically-correct conjugations were accepted. Subsequent analysis was done in order to extract number of words, number of times a participant used negation words (inter-rater correlation: r = 1), and number of times an aversive-aggressive concept or word was expressed (inter-rater correlation: r = 1). One participant was disqualified from analysis of the Story task due to a technical failure.

Object Use

Modifying a task employed by Back et al. (2009), participants were asked by an experimenter to state as many uses as they could for a brick. The time limit was two minutes. The experimenter wrote every usage the participant proposed, and subsequent analysis was done in order to extract number of uses, number of times a participant used negation words (inter-rater correlation: r = 1), and number of times an aversive-aggressive concept or word was expressed (inter-rater correlation: r = .99).

Numbers

Halverson (2003) conceptualized a highly conscientiousness person as being organized, achievement oriented, and not distractible. In order to tap Achievement Orientation, a task was devised in which on each trial two numbers (2-5 digits long) were presented simultaneously, on opposite sides of the computer monitor, for a duration of 250ms. The participants were instructed to determine whether the numbers were identical or not, using one of two designated response keys. Trial duration was 250ms, regardless of participant's response (i.e., 'no-response' was coded as an error). In between trials, a focus sign appeared for 200ms and a beep was sounded. No immediate feedback of success or failure was given to the participant. Trial-focal sign cycle went on for 180 seconds.

After three minutes, a notification appeared stating that the participants' success rate was 48%, and that the mean success rate for previous participants was 72%. These statistics were bogus, since only one of 49 subjects was able to respond within the 250ms time limit, and he was able to do so only twice (However, he was correct on both occasions). Participants were then offered the option to redo the Numbers task in order to improve their performance. Participants who chose not to redo the task continued to the next qIAT task. Participants opting to redo the task went through another cycle of 250ms trials, and 200ms focal-sign and beep, for three minutes.

After the additional three minutes, a notification appeared stating that the participants' success rate was 60%, and that the mean success rate for previous participants was 72%. These statistics were also bogus. Participants were then offered the option to redo the Numbers task once more. Participants who chose not to redo the task continued to the next qIAT task. Participants opting to redo the task once more were presented with a notification stating that, due to time constraints, the additional round will be held after the completion of all other tasks, if time permits. No further reference was made to the additional round of the Numbers task by the experimenter nor by a computerized notifications.

Participants' answers and response times were recorded, but more importantly, number of improvement intentions was recorded.

Triangles

Among others, three pairs of adjectives found by McCrae and Costa's (1987) factor analysis as prominent in the Conscientiousness dimension were 'careless – careful', 'Lazy – hard-working' and 'disorganized – well-organized'. In order to tap these aspects of the C dimension, a pen and paper task was devised in which participants were given a sheet of A4 paper with 1056 2.4mm X 2.4mm (approx.) geometrical shapes (circles, rectangles, rhombuses and triangles), organized in 22 columns and 48 rows (see Error! Reference source not found.). Shapes' pattern and scatter were identical across conditions. In order to create the illusion of a self-paced task, Participants were then told that they have to mark all, and only, the triangles on the sheet (a written example was given) and that the experimenter will return when the time ends. In addition, participants were told that if they were done before the experimenter returns they should open the door and call him, but were not told exactly how much time they have for the task. The experimenter then left the room, and started a clock, giving the subject one minute to complete the task before returning to the room.

A 'surface of processing' was calculated for each participant, based on the number of rows and columns she marked. Inside that surface, miss errors (i.e., non-marked triangles), as well as false-alarm errors (i.e., marked non-triangle shapes) were recorded. Scoring was done by dividing the sum of errors (i.e., miss and false-alarm) by the surface of processing. This was done in order to reduce the confound effect of speed of processing.

Due to a rigorous coding scheme (rows marked * columns marked = surface of processing), surface of processing, and therefore number of errors, was inflated for participants that switched strategies mid-task (i.e., started processing by rows, but changed to a columns-based strategy during the one minute period; n = 11). This feature makes the coding scheme somewhat biased against participants that switched strategies mid-task by penalizing them, and not calculating their actual surface of processing. However, it was determined a priori that such a penalty would yield an adequate score to participants whose over-all performance in the task was 'disorganized' and not 'planned-ahead'.

Self-introduction (SI)

Based on a task administered in previous research (Back et al., 2009), participants were seated in front of a video camera and requested to introduce and describe themselves: “During the next three minutes you are requested to describe yourself. After leaving the room [the experimenter] will start the clock, and will return to the room when the time ends. You are requested to address the camera, and use the entire time frame.”

Video footage was transcripted by the author and a colleague. Last names were replaced with three asterisk (***), but first names were transcripted. Subsequent analysis was done in order to extract number of words, number of times a participant used negation words (inter-rater correlation: r = 1), and number of times an aversive-aggressive concept or word was expressed (inter-rater correlation: r = .98). One participant was disqualified from analysis of the SI task due to a speech impairment (i.e., participant's speech was sequential but highly slurred) that prevented reliable transcription of text.

BIC

Participants were asked to complete a short version of the Behavioral Indicators of Conscientiousness Questionnaire (BIC), developed by Jackson et al. (2010). In the present study participants were given 15 items that Jackson et al. found to be highly correlated (; ) with explicit measurements of Conscientiousness (Jackson et al., 2010), and asked to indicate how often they took part in each behavior on a 1–5 scale, with responses ranging from 1) never performed the behavior to 5) performing the behavior quite often. Internal consistencies (Cronbach’s Alphas) in the current study were .76 for positive items (n = 6), .76 for negative items (n = 9) and .85 across all items (after reversed-items recoding). All items are presented in Table 2.

Facebook

Following a modified procedure of a task employed by Thalmayer et al. (2011), participants were asked if they would be willing to log in to their account using a computer in the lab, and state the number of their Facebook friends. In addition, as behavioral criterion for Extraversion, the experimenter noted the number of people in the participant's profile picture.

Instant Messaging (IM) Contacts

Modifying a task previously used by Thalmayer et al. (2011), participants were asked to count the number of contacts with whom they have exchanged text messages on the previous day, between 20:00-23:00. In order to avoid biases related to the weekend, participants who were tested on Sundays were asked to count the number of contacts with whom they have exchanged text messages on the previous Thursday, between 20:00-23:00. Since mobile phones have come a long way over the past few years in terms of technology, market penetration rate, and online availability, participants were asked to check for other chat services other than Short Term Messaging (SMS), such as Whatsapp, Facebook Chat, etc. In addition, asking whether the participants owned a phone seemed trivial, but participants not having a phone on them at the time of the experiment were recorded as such.

Table 2 Items of BIC questionnaire, with Cronbach's Alpha (across all items) if Item is deleted.

Item (n = 15, Cronbach's Alpha across all items = .845)

Cronbach's Alpha if Item Deleted

Keep my desk or work area clean

.837

Set a timeline for getting a project done

.829

Break daily routine [r]

.844

Finish a set amount of work before relaxing

.831

Work extra hard on a project to make sure that it is done right

.829

Complete the projects I start

.836

Put off work until the last minute [r]

.835

Lose something important in the clutter of my living quarters [r]

.837

Leave dirty clothes on the floor [r]

.826

Forget to write down important notes [r]

.825

Miss appointments [r]

.831

Turn in assignments late [r]

.846

Keep up with required work

.836

Borrow something and lose it, break it, or never return it [r]

.844

Forget materials for class/work [r]

.842

Call

Back et al. (2009) employed a task that tried to tap participants' inclination to help by asking them, as they were leaving the lab, to stay and help set-up another experiment for an additional 5 minutes. In order to tap the same trait, but avoiding additional time consuming tasks, a modified version of the task was employed in the present study. Ten minutes after being paid and leaving the lab, participants were called over the phone by the experimenter, and were told that due to a technical difficulty one part of the experiment's data was corrupted and lost. Participants were then asked to help by returning to the lab to repeat the lost part. If asked, the experimenter told the participants that they will not be reimbursed monetarily, nor with course credit, for repeating the experiment. If asked, the experimenter told the participants that the data corrupted belongs to one of the computerized questionnaires, but did not say which one. Participants' answers were recorded. Upon receiving a final answer, the experimenter told the participant: "I'm seeing that a colleague of mine was just able to restore the data. There is no need for you to come back. Thank you anyway."

Answers were coded on a nominal scale, with three groups: Yes, No, No-Answer. Therefore, participants' answers were simplified and analyzed by content (e.g., "I'll be there in 3 minutes," and "Yes, but I'm only available next week" were coded as 'Yes'. "I'll come if you can fix me up with more course credit" and "I've just left campus" were coded as 'No'. Reaching a voicemail service, or call-waiting was coded as 'No-Answer').

Post

Based on a task used by Back et al. (2009), participants were sent an email with a link to an online questionnaire and were told to complete it exactly one week from the beginning of the experiment session. Participants saying that they will not be able to do so at the one-week mark were asked to state a different time, as close as possible to the one-week mark. Participants' answers were recorded, as well as appointed and actual time of questionnaire completion. Difference between appointed and actual time was calculated in minutes, and was also recorded. Completion rate for the questionnaire was 65% (n = 32), subjects who did not complete the questionnaire were coded as such.

The questionnaire's items were adapted from a previous research (Grucza & Goldberg, 2007). Items were descriptions of different activities, and participants were asked to report the frequency with which they had carried out that activity, using the following response options: (1) Never in my life. (2) Not in the past year. (3) Once or twice in the past year. (4) Three or more times in the past year, but not more than 15 times (such as once or twice a month). (5) More than 15 times in the past year.

Items pertained to one of six clusters, based on the division made by Grucza and Goldberg (2007). However, an exeption was made, following Grucza and Goldberg (2007), for one item ("wrote poetry") that was associated with two clusters. The set included two clusters of relatively undesirable activities (here labeled Drug Use and Undependability), two clusters of relatively desirable activities (Friendliness and Creativity), and two clusters that seem relatively neutral in their desirability (Communication and Erudition). One item originally used by Grucza and Goldberg (2007; “Took a hard drug [for example, cocaine, LSD, or heroin]”) was removed for it seemed too extreme and it was feared that it might compromise the completion of the rest of the questionnaire. Another item "worked on a scrapbook" was also removed, but was replaced with a more contemporary activity, somewhat similar in nature ("edited a photograph [for example using an editing computer program]"). All items, divided by clusters, are presented in Appendix A, along with reliabilities for the current study.

Lexical

Participants completed an additional task similar in design to the Numbers task, but using Hebrew non-words (2-5 letters long) instead of numbers. In contrast with the Numbers task, participants were not presented with success rate feedback, and were not presented with an option to improve their score – after one 3 minutes cycle, participants continued to the next task. This task acted as a filler task between qIAT tasks, hence no data from this task was analyzed.

Aggregate Behavior Measures

Following Back et al. (2009), aggregate measures were calculated for some of the behaviors based on theoretical motives. Calculation was made by summing standard scores, except where standard scores were not applicable and manual scoring was required (Call task). Aggregate measures were calculated for negations usage (Number of negations used per word in Story and SI and number of negations used per use in Object Use), aversive-aggressive usage (Number of aversive-aggressive concepts or words used per word in Story and SI, and number of aversive-aggressive concepts or words used per use in Object Use), errors (number of errors in qIAT tasks, number of errors in Story task and number of errors in Triangles task). Additional aggregate measures were calculated for all behaviors that were hypothesized to be found in correlation with a specific Big-Five dimension but sample size was limited (n = 32) by the Post related variables. In order to increase sample size, Post related data was removed and new aggregate measures were calculated. However, reliability coefficients were not satisfactory (Cronbach's Alpha's mean = .13; SD = 0.36; max = .69; min = -.56), and aggregate measures were not used in data analysis.

Procedure

Participants completed the experiment individually. The majority of the experiment was conducted in an on-campus lab on a computer (qIAT, Big5, Story, Numbers, BIC, Lexical), while some were performed in front of an experimenter (Object, Facebook, SMS), one of the tasks was performed in front of a video camera (Self- Introduction), one was administered by pen and paper (Triangles), and one over the phone (Call). One additional task was completed on a computer, at a location chosen by the subject (Post).

Earlier versions of the IAT have shown to be fairly robust against explicit–implicit order effects (Hofmann et al., 2005). Yovel and Friedman (2012), however, chose to address a possible carryover effect from the implicit measurement (in which each item was presented many times) to the explicit measurement (in which items were presented only once) by having the self-report scales administered first. To further test the existence of a carryover effect, in the present study the order of administration between the qIAT implicit measurement and the Big5 explicit measurement was counterbalanced across participants.

In order to minimalize a possible practice effect, qIAT tasks were separated with behavior tasks. While the explicit-first/implicit-first was counterbalanced across participants, and order of the qIAT tasks was counterbalanced across participants using a Latin-square designed by Friedman (2012), the order of the behavior tasks was fixed across participants. For participants in the implicit-first conditions, the order was as follows: qIAT1, Story, qIAT2, Object, qIAT3, Numbers, qIAT4, Triangles, qIAT5, Lexical, Big5, Self-Introduction, BIC, Post (only briefing), Facebook, SMS. Participants were then debriefed, paid, and dismissed. Ten minutes after the participant's departure, the experimenter called her by phone (Call). Approximately a week later, participants completed an online questionnaire (Post). For participants in the explicit-first conditions, the order was as follows: Big5, Numbers, qIAT1, Story, qIAT2, Object, qIAT3, Lexical, qIAT4, Triangles, qIAT5, Self-Introduction, BIC, Post (only briefing), Facebook, SMS. Call, and Post procedures were identical. Without the time taken to complete the Post assignment, the session took 60-80 minutes. Participants’ behavior was videotaped throughout the course of the experiment.

Results

Descriptive Statistics

Means, standard deviations, and reliabilities of the personality measures can be found in Table 3. In addition, number of valid cases, means and standard deviations for all behavioral criteria and personality measures can be found in Appendix B. Internal consistencies proved satisfactory for all measures, with a mean coefficient alpha of .89 for explicit measures and .79 for implicit measures. Inter-correlations between personality measures were comparatively low for implicit measures (mean absolute inter-correlation of .19), but not so for the explicit measures (mean absolute inter-correlation of .35; see Table 4).

Preliminary analyses

In order to reduce the effect of outlying scores, we winsorized each variable, by replacing observations 2 standard deviations above the mean by observations at 2 standard deviations above the mean (Ratcliff, 1993). All following analyses were done using these scores. An alpha level of .05 was used in all statistical tests

In order to check if participant's gender had effect on the implicit and explicit measures of personality t tests were performed. Male participants (n = 22) had higher scores than female participants (n = 27) in the explicit measure of Conscientiousness (Mmale = 3.86, SDmale = 0.67; Mfemale = 3.25, SDfemale = 0.75; t(47) = 2.96, p < .05, d = 0.87). In addition, male participants had higher scores in the explicit measure of Neuroticism (Mmale = 3.54, SDmale = 0.67; Mfemale = 3.25, SDfemale = 0.75; t(47) = 3.79, p < .01, d = 1.11 ), as well as higher scores in the implicit measure of Neuroticism (Mmale = 0.27, SDmale = 0.34; Mfemale = .09, SDfemale = 0.27; t(47) = 2.06, p < .05, d = 0.6). Other tests on participants' gender did not reveal significant effects.

Multiple t-tests were performed to check if participant's choice of reimbursement (monetary or course credit) had effect on the implicit or explicit measures of personality, but did not reveal significant effects.

A t-test was performed to test if order of implicit-explicit measures had effect on the implicit and explicit measures of personality, but did not reveal significant effects. Furthermore, an analysis of variance did not show effect for the counterbalancing of the qIAT tasks' order on the implicit or explicit measures of personality.

Table 3 Descriptive Statistics of Explicit and Implicit Personality Predictors

Explicit

Implicit

Dimension

M

SD

α

M

SD

α

Neuroticism

3.11

0.81

.92

0.17

0.35

.84

Extraversion

3.18

0.77

.93

0.29

0.38

.81

Openness

3.52

0.77

.82

0.49

0.39

.81

Agreeableness

3.93

0.59

.89

0.52

0.33

.76

Conscientiousness

3.57

0.53

.91

0.55

0.38

.73

Note: Descriptives for explicit personality predictors are based on questionnaire data with a possible range from 0 to 5. Descriptives for implicit personality predictors are based on IAT data, using Greenwald, Nosek, and Banaji (2003) improved scoring algorithm.

Main Analyses

Convergent and discriminant validity of the qIAT

Supporting the convergent validity of the qIAT, significant explicit–implicit correlations were present for all dimensions addressed in the current study (rAgreeableness = .39, dAgreeableness = 0.85; rConscientiousness = .29, dConscientiousness = 0.60; rNeuroticism = .29, dNeuroticism = .60; see diagonal of Table 4 for all five dimensions). Supporting the discriminant validity of the qIAT, correlations between the implicit measure of Agreeableness and the explicit measures of Conscientiousness (r = .05) and Neuroticism (r = .14) were trivial-small. In addition, correlations between the implicit measure of Neuroticism and the explicit measures of Conscientiousness (r = .05) and Agreeableness (r = .16) were trivial-small. Furthermore, correlations between the implicit measure of Conscientiousness and the explicit measures of Neuroticism (r = .06) and Agreeableness (r = .22) were trivial-small. To conclude, convergent validity (Mean r for congruent implicit-explicit measures, Mrc = .32, SDrc = .05) and discriminant validity (Mean r for non-congruent implicit-explicit measures, Mrnc = .11, SDrnc = 0.67) of the qIAT was supported.

Table 4 Inter-correlations of Explicit and Implicit Personality Measures

Dimension

N

E

O

A

C

Neuroticism

.29*

-.06

-.03

.10

.25

Extraversion

.53**

.32*

.32*

.10

.30*

Openness

.54**

.42*

.07

.08

.35*

Agreeableness

.27

.25

.22

.39**

.34*

Conscientiousness

.31*

-.04

.79

.12

.29*

Note: Inter-correlations between implicit measures are shown above the diagonal, and inter-correlations between explicit measures are shown below it. Explicit-implicit correlations (in bold) are shown on the diagonal. * Correlation is significant at the .05 level (2-tailed). ** Correlation is significant at the .01 level (2-tailed).

Predictive Validity of Implicit and Explicit Measures

To examine the predictive validity of the implicit and explicit personality measures, correlations between each measure and the corresponding theoretically derived behavioral validation criterion were computed. Behavior measures that did not require T or ANOVA tests are presented in Table 5, along with number of valid cases. Findings are detailed separately for each dimension.

Conscientiousness: Correlations with the explicit measure were significant only for difference between actual and appointed Post completion time (r = -.43, p < .01, d = -.95), BIC questionnaire score (r = .75, p < .01, d = 2.27), Post questionnaire score – Undependability cluster (r = -.37, p < .05, d = -.80). Correlations with the implicit measure were significant only for difference between actual and appointed Post completion time (r = -.38, p < .05, d = -.82), BIC questionnaire score (r = .39, p < .01, d = 0.85), Post questionnaire score – Drug Use cluster (r = -.41, p < .01, d = -.90), and Number of errors in qIAT tasks (r = .25, p < .05, d = 0.52). All correlations were in the hypothesized directions, except for Number of errors in qIAT tasks. Mean of predictive validity of behavior for the implicit measure was small (M|r| = .22, SD|r| = 0.14). In addition, mean of predictive validity of behavior for the explicit measure was also small (M|r| = .22, SD|r|= 0.22).

Neuroticism: Correlations with the explicit measure were significant only for Number of aversive-aggressive concepts or words used in Object Use task, per use (r = -.35, p < .05, d = -.75). However, correlation found was not in hypothesized direction. No significant correlations were found with the implicit measure. Mean of predictive validity of behavior for the implicit measure was small (M|r| = .10, SD|r|= .06). In addition, mean of predictive validity of behavior for the explicit measure was also small (M|r| = .15, SD|r|= 0.10).

Agreeableness: Correlations with the explicit measure was significant only for Number of aversive-aggressive concepts or words used in SI task, per word (r = -.41, p < .01, d = -.90). Correlation found was in hypothesized direction. No significant correlations was found with the implicit measure. Mean of predictive validity of behavior for the implicit measure was trivial (M|r| = .07, SD|r|= .04). However, mean of predictive validity of behavior for the explicit measure was small (M|r| = .14, SD|r|= 0.12).

One-tailed planned comparisons were made to test for differences in the explicit and implicit measures of Conscientiousness and Agreeableness between three possible outcomes of Call task (Yes/No/No answer). It was hypothesized that participants that answered the call will have higher scores on both implicit and explicit measures of Conscientiousness and Agreeableness. Additionally, it was hypothesized that participants who agreed to come back to repeat a part of the experiment would have higher scores on both implicit and explicit measures of Conscientiousness and Agreeableness. However, one-tailed planned comparison made to test these hypotheses yielded no significant difference on explicit nor on implicit Conscientiousness and Agreeableness measures.

In addition, planned comparisons were made to test if the more a participant attempts to improve in Numbers task (0/1/2) the higher she will score on both measures of Conscientiousness. However, planned comparison between participant who chose to attempt improvement at least once and participants who did no attempts to improve yielded no significant difference on explicit nor on implicit Conscientiousness measures. Furthermore, planned comparison between participant who chose to attempt improvement once and participants who chose to attempt improvement twice yielded no significant difference on explicit nor on implicit Conscientiousness measures.

Table 5 Predictive validities of explicit and implicit measures

Dimension / Behavior measure

Explicit r

Implicit r

n

Conscientiousness

Difference between actual and appointed arrival time to the experiment (min)

.04

-.16

49

Absolute difference between actual and appointed arrival time to the experiment (min)

-.13

-.01

49

Difference between actual and appointed Post completion time (min)

-.43**

-.38*

32

Absolute difference between actual and appointed Post completion time (min)

-.19

-.23

32

Number of errors in qIAT tasks

.09

.25*

49

Number of errors in Story task

.05

.06

48

Number of errors in Triangles task

-.16

.07

49

BIC questionnaire score

.75**

.39**

49

Post questionnaire score – Drug Use cluster

.03

-.41**

32

Post questionnaire score – Undependability cluster

-.37*

-.23

32

Neuroticism

Number of uses in Object Use task

.27

.10

49

Post questionnaire score – Drug Use cluster

.11

.09

32

Number of negations used in Story task, per word

-.14

.05

48

Number of negations used in SI task, per word

-.11

.12

48

Number of negations used in Object Use task, per use

.06

.05

49

Number of aversive-aggressive concepts or words used in Story task, per word

.04

.05

48

Number of aversive-aggressive concepts or words used in SI task, per word

-.20

-.07

48

Number of aversive-aggressive concepts or words used in Object Use task, per use

-.35*

-.14

49

Difference between actual and appointed arrival time to the experiment (min)

.03

-.03

49

Absolute difference between actual and appointed arrival time to the experiment (min)

-.26

-.24

49

Difference between actual and appointed Post completion time (min)

-.08

-.17

32

Absolute difference between actual and appointed Post completion time (min)

.14

-.26

32

Agreeableness

Post questionnaire score – Friendliness cluster

.30

.10

32

Number of IM contacts

.16

.05

48

Number of Facebook friends

.16

.01

42

Number of aversive-aggressive concepts or words used in Story task, per word

.10

.06

48

Number of aversive-aggressive concepts or words used in SI task, per word

-.41**

.06

48

Number of aversive-aggressive concepts or words used in Object Use task, per use

-.01

.14

49

Difference between actual and appointed arrival time to the experiment (min)

-.11

.09

49

Absolute difference between actual and appointed arrival time to the experiment (min)

.05

-.02

49

Difference between actual and appointed Post completion time (min)

.13

-.11

32

Absolute difference between actual and appointed Post completion time (min)

-.01

.07

32

* Correlation is significant at the .05 level (2-tailed). ** Correlation is significant at the .01 level (2-tailed).

Behavioral criteria which were hypothesized to be in negative correlation with personality measure score are marked with italics.

A One-tail t-test was performed in order to check for differences in the explicit and implicit measures of Conscientiousness for participants that submitted Post questionnaire, and participants that failed to submit the Post questionnaire. While, as hypothesized, participants that submitted the questionnaire had significantly higher explicit Conscientiousness scores than participants who did not submit the questionnaire (Msubmitted = 3.69, SDsubmitted = 0.81; Mnot submitted = 3.21, SDnot submitted = 0.60; t(47) = -2.119, p < .05; d = 0.67), no significant difference was found in the implicit measure of Conscientiousness.

Discussion

In order to establish the validity and reliability of a measure to administer psychological questionnaires implicitly, the predictive, convergent and discriminant validity of an indirect assessment of personality (i.e., qIAT) were investigated in the present study, using a systematic and extensive behavioral approach. Additionally, the predictive validity of a direct measure of personality assessment (i.e., IPIP 50-items questionnaire) was investigated. Findings suggest that the convergent and discriminant validity of the qIAT were significant when validated against the explicit measures of assessment of Conscientiousness, Neuroticism and Agreeableness. However, the qIAT failed to exhibit sufficient predictive validity for the behavioral criteria defined in the present study. In a similar manner, predictive validity was low for the explicit measure of assessment of Conscientiousness, Neuroticism and Agreeableness.

Artificats and External Influence on the Measures of Personality

Findings in the current study with regard to an implicit-explicit carryover effect showed that the qIAT, as earlier versions of the IAT (Hofmann et al., 2005), was fairly robust against explicit–implicit order effects. This finding supports the construct validity of the qIAT, alleviating some doubts raised in Yovel and Friedman's (2012) research.

Gender differences in personality assessment found in the present study were not in accordance with existing literature. While a meta-analysis by Feingold (1994) found males to be higher than females on scores of Conscientiousness and lower than females on scores of Neuroticism, current results show opposite trends. Conflicting findings could be the result of the current study's sample size. While Feingold's meta-analysis reviewed a total of 159 independent samples (N = 36,459) the present study analyzed a very modest sample size (N = 49) of one independent sample. When compared with an extensive review as that of Feingold (1994) It is extremely likely that the effects of the current study are an anomaly.

Convergent and discriminant validity of the qIAT

While Yovel and Friedman's (2012) research focused solely on Extraversion, the present study expanded the scope the qIAT and applied it on Conscientiousness, Neuroticism and Agreeableness. As an extension to Yovel and Friedman's (2012) findings, convergent and discriminant validities were supported in the present study. Correlations between explicit and implicit measures of personality were not high (Mean r = .32), but independence of constructs between the implicit self-concept and the explicit self-concept, as suggested by Back, Schmukle, and Egloff's (2009) BPMP model, could account for the correlation's moderation (Hofmann et al., 2005).

Predictive Validity of Behavior for the Implicit Measure

While previous research showed that implicit measures of personality had good predictive validity (for recent examples, see: Asendorpf, Banse, & Mücke, 2002; Egloff & Schmukle, 2002; Nock et al., 2010; Schmukle et al., 2008; Yovel & Friedman, 2012), in the current study, predicative validity of the implicit measure was not significant for the grand majority of the delineated behavioral criteria. 77% (n = 10) of behavioral criteria for Conscientiousness were not reliably predicated by the congruent implicit measure (one behavior criterion, number of errors in qIAT tasks, was found significantly correlated, but not in the hypothesized direction) while 100% (n = 12) of behavioral criteria for Neuroticism and 100% (n = 11) of behavioral criteria for Agreeableness were not reliably predicated by the congruent implicit measure. In total, only 8% (n = 3) of the behavioral criteria employed for this report were predicted with significant validity, and in accordance with the research hypotheses.

Previous research suggests that spontaneous behavior, or "behavior that cannot be easily controlled voluntarily" (Steffens & Schulze König, 2006, p. 2), would be aptly predicted with an IAT-based measure (Schmukle et al., 2008; Steffens & Schulze König, 2006). However, two (i.e., BIC and Post – Drug Use) of the three behavioral criteria that were reliably predicted pertain to explicit self-report questionnaires about behavior. While it is difficult to argue that completing a self-paced multiple-choice questionnaire is a highly spontaneous behavior, other behavioral criteria, more theoretically attuned to the definition of spontaneous behaviors (e.g., Number of errors in Triangles task, Number of errors in Story task or submission of the Post questionnaire), did not reveal significant effects. The findings of the present study differ from existing literature, but due to the staggeringly low percentage of reliable predictions made, and since spontaneity was not a key concept in a priori delineating behavioral criteria, theoretical implications about the nature of predictable behaviors should not be made until further research is accomplished.

In addition, two of the three predictable tasks (i.e., BIC and Post) were explicit self-report questionnaires about behavior, not direct observations of behavior. Only one of the three reliably predictable measures, Difference between actual and appointed Post completion time, was truly a direct observation of behavior. The present study's behavioral criteria are problematic for they do not address an issue formulated recently by Back et al. (2009) – self-reported behavior (asking people what they do) and actual behavior (observing what people do) are not conceptually identical. In fact, this theorem is the keystone of the BPMP model.

Predictive Validity for the Explicit Measure and Quality of the Behavior Criteria

While previous research showed that explicit measures of personality had strong predictive validity (for recent examples, see: Back et al., 2006, 2009; Thalmayer et al., 2011), in the current study, predicative validity of the explicit measure was not significant for the majority of the delineated behavioral criteria: 69% (n = 9) of behavioral criteria for Conscientiousness were not reliably predicated by the congruent explicit measure; 91% (n = 10) of behavioral criteria for Agreeableness were not reliably predicated by the congruent explicit measure; 100% (n = 12) of behavioral criteria for Neuroticism were not reliably predicated by the congruent explicit measure (the only significant correlation, Number of aversive-aggressive concepts or words used in Object Use task, per use, was not in the hypothesized direction). In total, of the behavioral criteria employed for this report, only 14% (n = 5) were predicted with significant validity.

The current study's failure to demonstrate significant predictive validity of behavior for the explicit measure, combined with the lack of predictive ability of the implicit measure, and the low internal consistency of the behavior aggregates, raises doubts as to the definitions of the behavioral criteria themselves. While behavioral criteria were based on paradigms developed in previous research, or on theoretical reasoning with a narrow and conservative interpretation of concepts, it is possible that flaws in operationalization and implementation, or mistakes – theoretical or technical – made by the author in adapting certain tasks, hindered revealing statistical significance to acquire higher predictive validity.

Limitations, Theoretical Implications and Future Research

Using the BPMP (Back et al., 2009) as its theoretical framework, the current study was aimed at demonstrating that implicit and explicit measures of personality assessment can predict, independently, actual behavior, thus supporting the notion of an independent, but not unrelated, implicit self-concept and explicit self-concept. However, the present study could not establish predictive validity of behavior for the qIAT, and importantly, could not establish predictive validity for the extensively researched and strongly validated explicit measure (Back et al., 2006, 2009; Thalmayer et al., 2011). While it could be argued that the current study rebutted a long-standing research tradition, it seems prudent, and more plausible, to assume that the limitations of the current study prevent it from being taken into theoretical consideration.

As noted above, sample size for the present study was very limited. For this reason, it is very likely that the current study lacked statistical power to point out effects that would have been otherwise significant. This flaw is cardinal, for it affects not only effects that might have been overlooked; any effect found significant in the present study deserves a review and requires replication on a larger sample size that would provide better estimations of the population. This study's findings are therefore dimmed inconclusive and further research is required to obtain a more in-depth insight into the qIAT qualities as an indirect assessment tool of personality, and as a predictor of behavior. Increasing sample size, and developing better behavioral criteria and better aggregated behavioral criteria would be essential for future research. Behavioral criteria in future research should focus less on self-reports of behavior (e.g., BIC and Post questionnaires) and more on direct observation of behavior (e.g., analysis of video recording). While such data was collected for the present study, it was removed from analysis due to lack of expert raters and resources.

Additionally, although the present research addressed, and with satisfactory results, the convergent and discriminant validity of the qIAT on Conscientiousness, Agreeableness and Neuroticism, future research should concentrate on other aspects of the qIAT's reliability, such as test-retest reliability and parallel-forms reliability. For example, successfully using the qIAT to measure personality using the HEXACO-60 (Ashton & Lee, 2009) items pool, or even the BIC (Jackson et al., 2010) or Post (Grucza & Goldberg, 2007) questionnaires, would greatly support the parallel-forms reliability and over-all validity of the qIAT as a tool to predict behavior and indirectly measure constructs that have been measured only explicitly, or using single-word IAT, until now.

BIB

Asendorpf, J. B., Banse, R., & Mücke, D. (2002). Double dissociation between implicit and explicit personality self-concept: The case of shy behavior. Journal of Personality and Social Psychology, 83(2), 380–393. doi:10.1037//0022-3514.83.2.380

Ashton, M. C., & Lee, K. (2009). The HEXACO-60: a short measure of the major dimensions of personality. Journal of personality assessment, 91(4), 340–5. doi:10.1080/00223890902935878

Back, M. D., Schmukle, S. C., & Egloff, B. (2006). Who is late and who is early? Big Five personality factors and punctuality in attending psychological experiments. Journal of Research in Personality, 40(5), 841–848. doi:10.1016/j.jrp.2005.11.003

Back, M. D., Schmukle, S. C., & Egloff, B. (2009). Predicting actual behavior from the explicit and implicit self-concept of personality. Journal of personality and social psychology, 97(3), 533–48. doi:10.1037/a0016229

Bluemke, M., & Fiedler, K. (2009). Base rate effects on the IAT. Consciousness and cognition, 18(4), 1029–38. doi:10.1016/j.concog.2009.07.010

Bosson, J. K., Swann, W. B. . J., & Pennebaker, J. W. (2000). Stalking the perfect measure of implicit self-esteem: The blind men and the elephant revisited? Journal of Personality and Social Psychology, 79(4), 631–643. doi:10.1037//0022-3514.79.4.631

Caspi, A., Roberts, B. W., & Shiner, R. L. (2005). Personality development: stability and change. Annual review of psychology, 56, 453–84. doi:10.1146/annurev.psych.55.090902.141913

Dijksterhuis, A., & Nordgren, L. F. (2006). A Theory of Unconscious Thought. Perspectives on Psychological Science, 1(2), 95–109. doi:10.1111/j.1745-6916.2006.00007.x

Egloff, B., & Schmukle, S. C. (2002). Predictive validity of an implicit association test for assessing anxiety. Journal of Personality and Social Psychology, 83(6), 1441–1455. doi:10.1037//0022-3514.83.6.1441

Feingold, a. (1994). Gender differences in personality: a meta-analysis. Psychological bulletin, 116(3), 429–56. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7809307

Feuerhahn, N., Kühnel, J., & Kudielka, B. M. (2012). Interaction effects of effort–reward imbalance and overcommitment on emotional exhaustion and job performance. International Journal of Stress Management, 19(2), 105–131. doi:10.1037/a0028338

Freud, S. (n.d.). Mourning and melancholia, XIV. Retrieved from http://www.pep-web.org/document.php?id=SE.014.0237A

Goldberg, L. R. (2005). Possible Questionnaire Format for Administering the 100-Item Set of IPIP Big-Five Factor Markers. Retrieved September 26, 2012, from http://ipip.ori.org/New_IPIP-100-item-scale.htm

Gray, N. S., Brown, A. S., MacCulloch, M. J., Smith, J., & Snowden, R. J. (2005). An implicit test of the associations between children and sex in pedophiles. Journal of abnormal psychology, 114(2), 304–8. doi:10.1037/0021-843X.114.2.304

Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review, 102(1), 4–27. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7878162

Greenwald, A. G., & Farnham, S. D. (2000). Using the Implicit Association Test to measure self-esteem and self-concept. Journal of Personality and Social Psychology, 79(6), 1022–1038. doi:10.1037//0022-3514.79.6.1022

Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: the implicit association test. Journal of personality and social psychology, 74(6), 1464–80. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9654756

Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and using the Implicit Association Test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85(2), 197–216. doi:10.1037/0022-3514.85.2.197

Grucza, R. A., & Goldberg, L. R. (2007). The comparative validity of 11 modern personality inventories: predictions of behavioral acts, informant reports, and clinical indicators. Journal of personality assessment, 89(2), 167–87. doi:10.1080/00223890701468568

Halverson, C. F., Havill, V. L., Deal, J., Baker, S. R., Victor, J. B., Pavlopoulos, V., Besevegis, E., et al. (2003). Personality structure as derived from parental ratings of free descriptions of children: the inventory of child individual differences. Journal of personality, 71(6), 995–1026. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/14633056

Hofmann, W., Gawronski, B., Gschwendner, T., Le, H., & Schmitt, M. (2005). A meta-analysis on the correlation between the implicit association test and explicit self-report measures. Personality & social psychology bulletin, 31(10), 1369–85. doi:10.1177/0146167205275613

Jackson, J. J., Wood, D., Bogg, T., Walton, K. E., Harms, P. D., & Roberts, B. W. (2010). What do conscientious people do? Development and validation of the Behavioral Indicators of Conscientiousness (BIC). Journal of Research in Personality, 44(4), 501–511. doi:DOI 10.1016/j.jrp.2010.06.005

McCrae, R. R., & Costa, P. T. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of personality and social psychology, 52(1), 81–90. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/3820081

McCrae, R. R., & John, O. P. (1992). An introduction to the five-factor model and its applications. Journal of personality, 60(2), 175–215. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/1635039

Nock, M. K., Park, J. M., Finn, C. T., Deliberto, T. L., Dour, H. J., & Banaji, M. R. (2010). Measuring the suicidal mind: implicit cognition predicts suicidal behavior. Psychological science, 21(4), 511–7. doi:10.1177/0956797610364762

Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46(3), 598–609. doi:10.1037//0022-3514.46.3.598

Pennebaker, J. W., & Chung, C. (2007). The development and psychometric properties of LIWC2007. … , TX, LIWC. Net, 1–22. Retrieved from http://homepage.psy.utexas.edu/homepage/faculty/pennebaker/Reprints/LIWC2007_LanguageManual.pdf

Raad, B. De. (2006). An expedition in search of a fifth universal factor: Key issues in the lexical approach. European Journal of Personality, 8(4), 229–250. doi:10.1002/per.2410080403

Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychological bulletin, 114(3), 510–32. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/8272468

Rohner, J. C., & Björklund, F. (2006). Do self-presentation concerns moderate the relationship between implicit and explicit homonegativity measures? Scandinavian journal of psychology, 47(5), 379–85. doi:10.1111/j.1467-9450.2006.00522.x

Sartori, G., Agosta, S., Zogmaister, C., Ferrara, S. D., & Castiello, U. (2008). How to accurately detect autobiographical events. Psychological science, 19(8), 772–80. doi:10.1111/j.1467-9280.2008.02156.x

Schlesinger, L. (2009). Psychological profiling: Investigative implications from crime scene analysis. J. Psychiatry & L. Retrieved from http://heinonlinebackup.com/hol-cgi-bin/get_pdf.cgi?handle=hein.journals/jpsych37&section=9

Schmukle, S. C., Back, M. D., & Egloff, B. (2008). Validity of the Five-Factor Model for the Implicit Self-Concept of Personality. European Journal of Psychological Assessment, 24(4), 263–272. doi:10.1027/1015-5759.24.4.263

Shiner, R. L. (1998). How shall we speak of children’s personalities in middle childhood? A preliminary taxonomy. Psychological Bulletin, 124(3), 308–332. doi:10.1037//0033-2909.124.3.308

Steffens, M. C., & Schulze König, S. (2006). Predicting Spontaneous Big Five Behavior with Implicit Association Tests. European Journal of Psychological Assessment, 22(1), 13–20. doi:10.1027/1015-5759.22.1.13

Thalmayer, A. G., Saucier, G., & Eigenhuis, A. (2011). Comparative validity of brief to medium-length Big Five and Big Six Personality Questionnaires. Psychological assessment, 23(4), 995–1009. doi:10.1037/a0024165

Wilson, T. D. (2009). Know Thyself. Perspectives on Psychological Science, 4(4), 384–389. doi:10.1111/j.1745-6924.2009.01143.x

Yovel, I., & Friedman, A. (2012). Bridging the gap between explicit and implicit measurement of personality: The questionnaire-based implicit association test. Personality and Individual Differences. doi:10.1016/j.paid.2012.08.015

Appendix

Appendix A Post questionnaire items, divided by clusters, with Cronbach's Alpha intra-cluster. Inter cluster Cronbach's Alpha = .84.

Friendliness (n = 8, Cronbach's Alpha = .85)
Item Cronbach's Alpha if Item Deleted
Apologized to someone

0.84

Started a conversation with strangers

0.84

Made a new friend

0.85

Shared a problem with a close friend or relative

0.84

Complimented someone

0.83

Did a favor for a friend

0.82

Hugged someone

0.83

Was consulted for help or advice by someone with a personal problem

0.83

Drug Use (n = 12, Cronbach's Alpha = .84)
Item Cronbach's Alpha if Item Deleted
Drank alcohol or used other drugs to make myself feel better

0.83

Had an alcoholic drink before breakfast or instead of breakfast

0.85

Drank whiskey, vodka, gin, or other hard liquor

0.82

Became intoxicated

0.82

Smoked Marijuana

0.85

Drank wine

0.83

Drove a car after having a few alcoholic drinks

0.84

Drank alcohol during working hours

0.84

Had a hangover

0.83

Drank in a bar

0.82

Went to a nightclub

0.85

Drank beer

0.83

Undependability (n = 7, Cronbach's Alpha = .75)
Item Cronbach's Alpha if Item Deleted
Let work pile up until just before a deadline

0.73

Changed or canceled an appointment

0.75

Did not return a phone call

0.72

Broke a promise

0.71

Borrowed something and lost it, broke it, or never returned it

0.71

Misplaced something important (glasses, car keys, etc.)

0.74

Arrived at an event more than an hour late

0.68

Creativity (n = 11, Cronbach's Alpha = .74)
Item Cronbach's Alpha if Item Deleted
Played in or conducted a band or orchestra

0.74

Wrote poetry

0.72

Talked in a language other than Hebrew

0.70

Played a piano or other instrument

0.72

Acted in a play

0.74

Gave a prepared talk or public recital (vocal, instrumental, etc.)

0.72

Sang in or conducted a choir or small ensemble

0.73

Took music lessons (voice or instrument)

0.73

Asked questions in a meeting or lecture

0.75

Painted a picture (oil, watercolor, pastel, etc.)

0.71

Produced a work of art

0.68

Erudition (n = 6, Cronbach's Alpha = .66)
Item Cronbach's Alpha if Item Deleted
Went to a public library

0.62

Read in bed before going to sleep

0.58

Bought a book

0.57

Read a book

0.62

Read an entire book in one sitting

0.69

Had an overdue fine for a movie rental or library book

0.63

Communication (n = 8, Cronbach's Alpha = .62)
Item Cronbach's Alpha if Item Deleted
Wrote a thank-you note

0.55

Wrote poetry

0.63

Wrote a handwritten letter

0.56

Read poetry

0.56

Put pictures in a photo album

0.58

Wrote a postcard

0.63

Edited a photograph (for example using an editing computer program)

0.65

Appendix B Number of Valid Cases, Means and Standard Deviations for all Behavioral Criteria

Variable

N

Mean

Std. Deviation

Age

49

24.92

2.33

BIC: Questionnaire score

49

3.83

0.41

Facebook: Number of friends

42

331.17

207.20

Facebook: Number of people in profile picture

37

1.30

0.62

IM: Number of IM contacts

48

3.18

5.19

Numbers: Number of improvement attempts

49

1.33

0.85

Object Use: Number of aversive-aggressive concepts or words

49

1.05

1.06

Object Use: Number of aversive-aggressive, per objects uses

49

11.79

12.10

Object Use: Number of negations used in task

49

0.24

0.48

Object Use: Number of negative words, per object uses

49

2.86

6.01

Object Use: Number of uses in Object Use task

49

9.55

4.50

Post: Questionnaire score – Communication cluster

32

2.52

0.60

Post: Questionnaire score – Creativity cluster

32

2.37

0.51

Post: Questionnaire score – Drug Use cluster

32

2.65

0.67

Post: Questionnaire score – Erudition cluster

32

2.95

0.56

Post: Questionnaire score – Friendliness cluster

32

4.16

0.55

Post: Questionnaire score – Undependability cluster

32

2.67

0.56

qIAT: Average reaction-time (millisecond)

49

1159.03

269.14

qIAT: Total number of errors

49

78.10

42.47

SI: Average number of letters in word

48

4.31

0.27

SI: Number of aversive-aggressive concepts or words

48

2.70

3.17

SI: Number of aversive-aggressive, per words

48

1.73

1.65

SI: Number of negations used in task

48

3.10

3.08

SI: Number of negative words, per words

48

1.96

1.81

SI: Number of words

48

156.96

81.33

Story: Average number of letters in word

48

4.61

0.19

Story: Number of aversive-aggressive concepts or words

48

4.05

2.55

Story: Number of aversive-aggressive, per words

48

4.17

2.35

Story: Number of errors

48

0.48

0.71

Story: Number of negations used in task

48

2.08

1.85

Story: Number of negative words, per words

48

2.18

2.08

Story: Number of words

48

99.13

31.03

Time: Absolute difference between actual and appointed arrival time to the experiment (min)

49

3.82

4.82

Time: Absolute difference between actual and appointed Post completion time (min)

32

1328.16

2231.41

Time: Difference between actual and appointed arrival time to the experiment (min)

49

-1.33

6.02

Time: Difference between actual and appointed Post completion time (min)

32

558.34

2545.25

Triangles: Number of columns marked

49

22.00

.00

Triangles: Number of error, by surf of processing

49

4.52

5.87

Triangles: Number of errors (miss+fa)

49

34.53

63.29

Triangles: Number of marked non-triangles

49

.06

0.24

Triangles: Number of non-marked triangles

49

34.47

63.32

Triangles: Number of rows marked

49

22.10

12.88

Triangles: Surface of processing

49

486.24

283.32

 

Appendix C Correlations for Implicit and Explicit Measures, for all Personality Dimensions

Correlations

Implicit A

Implicit E

Implicit C

Implicit N

Implicit O

Explicit A

Explicit E

Explicit C

Explicit N

Explicit O

Implicit A

1

.102

.337*

.103

.085

.392**

.048

.047

.140

.285*

Implicit E

.102

1

.305*

-.062

.325*

.366**

.320*

.149

.184

.303*

Implicit C

.337*

.305*

1

.250

.350*

.220

.050

.286*

.056

.307*

Implicit N

.103

-.062

.250

1

-.032

.161

.039

.045

.286*

.279

Implicit O

.085

.325*

.350*

-.032

1

.118

-.066

.128

-.056

.070

Explicit A

.392**

.366**

.220

.161

.118

1

.254

.122

.265

.219

Explicit E

.048

.320*

.050

.039

-.066

.254

1

-.045

.528**

.418**

Explicit C

.047

.149

.286*

.045

.128

.122

-.045

1

.311*

.079

Explicit N

.140

.184

.056

.286*

-.056

.265

.528**

.311*

1

.542**

Explicit O

.285*

.303*

.307*

.279

.070

.219

.418**

.079

.542**

1

*. Correlation is significant at the .05 level (2-tailed).**. Correlation is significant at the .01 level (2-tailed).

Appendix D Sample of Triangle task

tri

מודעות פרסומת
להגיב

להשאיר תגובה

הזינו את פרטיכם בטופס, או לחצו על אחד מהאייקונים כדי להשתמש בחשבון קיים:

הלוגו של WordPress.com

אתה מגיב באמצעות חשבון WordPress.com שלך. לצאת מהמערכת / לשנות )

תמונת Twitter

אתה מגיב באמצעות חשבון Twitter שלך. לצאת מהמערכת / לשנות )

תמונת Facebook

אתה מגיב באמצעות חשבון Facebook שלך. לצאת מהמערכת / לשנות )

תמונת גוגל פלוס

אתה מגיב באמצעות חשבון Google+ שלך. לצאת מהמערכת / לשנות )

מתחבר ל-%s

%d בלוגרים אהבו את זה: