Category Archives: Evaluation

RESEARCH QUESTIONS

The following  are the research questions that we have for our project, keeping in mind our GOALS of the research for Exploring Interactive Theatre. We have formulated these questions that fit our goals and our research.

1.1  What is interactive theater?

1.2

1.2.1  How is the performance of the actors influenced by the interactive parts?
How hard/easy is it to learn how to use it? Are they comfortable with the usage? Are there any disturbances between different parts? Do they have any tips, how to improve it?

1.2.2  How did the cooperation between designers and technicians influence the creative output? What kind of problems appeared? Was there anything that made working together harder
or easier?

1.2.3  Does the audience realize the interactive parts?
Are they able to get the interaction? Are they interested in how it works? Do they like it?
Does the audience get the whole concept of the play?

2

2.1  Are there conventional techniques that could be an alternative solution for the interactive parts?
Where does conventional theater end and interactive theater start? What are the differences,
what are the similarities between conventional and interactive theatre?

2.2  What is the motivation behind putting interactive props/costumes on stage?
What is the biggest advantage of using interactive technologies in that theater play? Is the usage of interactive parts enhancing the outcome of the play?

2.3  What do actors/audience/creators expect when they take part in an interactive theater play?
Where the interactive parts distracting or did the actors/audience like it?
How would actors/audience/creators participating in the project describe the term interactive theater?

Methods For Evaluation

                 ACTORS                AUDIENCE  CREATORS
               Semi – structured                               Interview                            (with Video & Audio)*      Semi – structured                       Interview** Semi – structured   Interview**
„Not-call-it -Focus-Group“-
Discussion
(with Video & Audio)**
“Audience Talk”
Publikumsgespräch
(record Audio)
Focus Group:divided in Tec.
and Design groups
(record Audio)**
——- Questionnaires ——-
Observation In field Observation In field ——-
Direct(notes) Direct(notes)
Indirect(camera during rehersals and play)

 

* done by Project-teams

** optional

 

Actors:

Semi-Structured Interview:

Our method of evaluation for the actors is a semi-structured interview, where its an open conversation but we have a set of guiding questions to cover the same topics with each participant. Questions are open and closed. Interviewer follows a framework of questions but can react to the participants individually.  By this method we are able to explore deeper into the questions and the  best way to understand the  experience of actors and experts (director, stage designer).

Observation:

In this observation method we directly observe the actors  experience and reactions to the interactive technology when they are rehearsing in their natural environment (in field) and also during the final play. We instantly observe situations and take notes  about the reaction of actors in rehearsal; thereby making the analysis suddenly(direct). In the final, (indirect) observation, we collect data about the development of the play, where the creators take notes of their activities on a regular basis; instrumenting the software to record users’ activity in a log; video-recording during rehearsals and plays.

Audience:

Questionnaire:

By using the Questionnaire method we get answers to specific questions from a large group of people, useful to confirm the conclusions from the audience after watching the play. Closed or open questions can be used to collect demographic data and users’ opinions.

“Audience Talk”:Publikumsgespräch

In the Publikumsgespräch, the audience can ask questions to the the actors, director and the creators. It is an open talk session where all question are open regarding the play, for example, questions to the actors about their acting, about the story to the director, technical questions to the creators and so on.

Observation:

In the Field observation we directly observe the audience and their reaction to the play in their natural environment. In the direct observation we instantly observe situations and take notes  about the reaction of the audience during the play; thereby making the analysis suddenly. In the indirect observation we collect diaries (=notes), interaction logs, photos and videos material to analyze later, which is useful to track the audience’s activities during the play.

 

Data Gathering Techniques

Rogers, Y., Sharp, H., Preece, J. Interaction design : beyond human-computer interaction. 3rd ed., Wiley, 2011,  p. 222-268.

Technique Definition Kind of data Relevance to our project
Interviews “Conversation with purpose” Mostly Qualitative Some Quantitative Exploring issues
Unstructured Exploratory Conversation around one topic,Questions are open. Qualitative Can be used to explore the range of opinions, good for deeper understanding of the topic.→ first questions to the actors
Structured Predetermined conversation with reproducible course. Questions are closed, short and clearly worded. Every participant is asked the same and in the same order. Qualitative and Quantitative Working well when people are in a rush since they only need to answer specific questions.→ actors and audience after seeing play.
Semi-structured A guided conversation to cover the same topics with each participant Questions are open and closed. Interviewer follows a framework of questions but can react to the participants individually. Qualitative and Quantitative Is able to explore deeper questions as well as to cover certain topics→ best way to track down experience of actors and experts (director, stage designer)
Focus Groups Conversation of a small group guided by one trained facilitator. Participants represent target population. Questions are open and lead to a flexible discussion with agenda that it follows. Mostly Qualitative Some Quantitative Collecting multiple viewpoints, explores also sensitive or diverse questions, good for community issues. Encourages contact between developers and actors.→ overview of actors opinions, group experience
Questionnaires Use closed or open questions to collect demographic data and users’ opinions. Qualitative and Quantitative Getting Answer to specific questions from a large group of people, useful to confirming the conclusions.→ audience after watching the play
Observation Paying close attention to the users in order to gather data required. Qualitative and Quantitative Gathering data at different/any stage during the development of the play.
In the field Getting a full and true story through observing what the users are doing or how to achieve a task in their natural environment Mostly Qualitative Filling in details and nuances in that are not elicited from the other forms of investigation.→ actors during the rehearsals; creators when designing the play; audience when watching the play
Controlled environment Monitoring users within “a purpose-built usability laboratory” (what they are doing is within an artificial situation) Qualitative and Quantitative Researching on the details of what individuals do in the laboratory.→ inviting actors to the laboratory to test equipment; inviting audience to the rehearsals
Indirect Collecting material to analyze later.Useful to tracking users’ activities that cannot be present over the duration of the study.Diaries (=notes), interaction logs, photos and videos are several main techniques. Qualitative and Quantitative Collecting data about the development of the play.→ creators take notes of their activities on a regular basis; instrumenting the software to record users’ activity in a log; video-recording during rehearsals and plays
Direct Instantly observing situations and taking notes etc.Making analysis suddenly. Qualitative and Quantitative Collecting data about certain situations.→ observing and taking notes about the reaction of actors in rehearsal; the reaction of audience during the play

 

 

Exploring Interactive Theatre – The Goals


The goal of the research is to explore Interactive Theatre. In achieving this goal, the commission will address two strategic objectives. The following index will give more details about what that means and how it is planned to setup the study.

1. Explore the effects of interactive technology involved.
The commission wants to find out about the feelings and thoughts that people have participating in the Interactive Theatre. To cover every perspective three different populations will be analyzed.

1.1 Find characteristics, despriptions and a definition for interactive effects applied in theater plays

1.2 Analyze the feelings and thoughts that people have being involved in Interactive Theatre. Considering three different populations:
1.2.1 The actors, the people that use the interactive technology on stage and for that reason will be the main source of direct feedback.
1.2.2 The creators, the people developing and thinking about the usage of interactive technology and therefore will be some sort of experts.
1.2.3 The audience, the people who experience the interactive theater play and how it affects their viewer experience.

2. Explore the impact that the interactive technology has on the play
The commission will measure the impact of interactive technology so as to see to what extent has the technology influence the theater.
To attain the objective, the research will cover three aspects:

2.1 The impact of interactive technology in comparison to conventional staging techniques
A comparison between conventional and interactive theater will be made to extract differences

2.2 The necessity of using interactive technology in theater
Finding out the necessity of using interactive technology in theater as well as the additional benefits of involving interactive technology into a play

2.3 The perception of interactive theater
Investigating the perception of Interactive Theater through participants (users, secondary stakeholder, audience) feedback

 


Our understanding of how to evaluate interactive media is mainly based on “Interaction Design – beyond human-computer interaction 3rd Edition”, Yvonne Rogers, Helen Sharp, Jennifer Preece, published by John Wiley & Sons (2011).

For more information about the explicit source, take a look at the post “Data Gathering Techniques”

D.E.C.I.D.E. !

Rogers, Y., Sharp, H., Preece, J. Interaction design : beyond human-computer interaction. 3rd ed., Wiley, 2011, pp. 456 ff.

DECIDE: A Framework to Guide Evaluation

1. Determine the goals

2. Explore the questions

3. Choose the evaluation methods

4. Identify the practical issues

5. Decide how to deal with the ethical issues

6. Evaluate, analyze, interpret, and present the data.”

According to this framework, we can design, plan, and conduct our study. Our main goals have been determined Exploring Interactive Theater – The Goals. Based on that the question are to be asked by the mini-project groups (blog post for “Research Questions” can be found here). Together we can than decide which method or combination of methods are appropriate to get answers to our research questions. A post about possible methods & their relevance for our project can be found Data Gathering Techniques. After that we can continue with step 4 & 5: We can filter the practical issues based on the design-tech development processes & the methods to apply. Furtehrmore, we can take ethical concerns into account. The last main part is the evaluation itself – firstly, we have to prepare e.g. questions & guidelines for interviews or we have to setup cameras for observation purposes, then we will collect the data during the rehearsals & plays. Afterwards, we can analyze them & make interpretations for publishing our exploration results.