Lessons from the VIPROM-Trainings – Results from a Multi-Perspective Evaluation

Image by rawpixel.com on Freepik

Within the VIPROM-project the Institute of Conflict Research (IKF Vienna) had the task of evaluating both the content of the learning platform and the pilot trainings taking place in all partner countries. To be completely honest, for a project-financed social science institute doing evaluations is something of a bread-and-butter job: Yes, it is undoubtedly important. Yes, we are good at it. But it is not the most exciting thing we do – well… normally. Every once in a while, however, a project comes along that proves to be special. And the VIPROM-project turned out to be very special. This was not just because countering domestic violence is a topic of huge importance (which is what motivates us to do the best we can every day); it was not just because the people involved were passionate about their work (even though that is the first condition for success); it was not even just because it was well-designed and thoroughly planned in terms of efficiency and impact (as projects have to be). Rather, it was one of these rare projects, where societal importance, passion, meticulous planning and execution converged into something extraordinary. And after having actually done the evaluation, we can proudly say that we now have the evidence to back up this claim.

WHAT A MULTI-PERSPECTIVE EVALUATION TAUGHT US

Obviously, when you plan your work, you don’t know how it is going to turn out. As social scientists we planned our evaluation to the best of our knowledge and experience. Evaluating trainings might at first glance seem a straight-forward task – all you really want to find out is whether people know more after they participated in the training than they knew before, right? – but in practice it gets quite complicated. How do you actually assess these knowledge gains, especially when you’re dealing with professionals who you can hardly submit to an hour-long exam? And even more challenging: How do you measure gains in practical competencies? How do you assess that training participants did not exclusively amass theoretical knowledge, but that the training will translate into real-world changes? An international environment does not make this easier – how will you know, whether someone whose language you don’t even speak is a more competent and emphatic listener when dealing with domestic violence victims after the training? The answers we found in the evaluations of the VIPROM-training can be boiled down to two main strands: a) get help from your project-partners (translations, observations, feed-back and advice…) and b) go for a multi-method approach. 

The latter meant that we combined quantitative and qualitative data, took diverse perspectives into account and tried to counter known difficulties through a combination of several complimentary approaches. In effect, we did participant surveys at different points in time with a bit of an extra twist (i.e. not stopping at participants’ opinions but trying to include objectified measurements of knowledge gains), combined those with trainers’ perspectives, included training observations and expert opinions.

THE VIPROM TRAININGS ARE A SUCCESS STORY!

Shortly (by late-November) a deliverable will be available on the VIPROM-website detailing our findings and presenting data and facts. Some of our fascinating findings include data on how the VIPROM-trainings managed to achieve sustainable learning on an individual level and how they might even have managed to trigger institutional change as well as results concerning remaining challenges.

But some of the most important lessons are to be found in the details and connections that are hard to measure in the strict sense of the term. In hindsight, results corroborate the inclusion of qualitative methods into the evaluation strategy more strongly than any abstract epistemological argument could.  For example, we are proud to share that after taking part in the VIPROM-program medical students (n=267) rated the overall quality of the training at 8,63 on a 1 to 10 scale and medical professionals (n=197) even at a whooping 9,15. Maybe even more important, when surveyed again six months after completing the training medical professionals rated the statement that the training “positively changed some aspects of their approach and communication with victims of domestic violence” at 8,33 (n=133, scale from 1 to 10) and that it “supports their efforts in driving forward changes in their area of work that benefit victims of domestic violence” at 8,44 (n=116, scale from 1 to 10). These numbers come to life when complemented by qualitative data, which helps explain how this success was possible. For example, observers in some locations found that trainees were sitting together in breaks and after the sessions plotting on how to get their colleagues involved in DV-related work; medical students used open fields in the surveys to express their gratitude for having encountered a safe space in which to express not only experiences, but also discuss difficult questions; seasoned professionals remarked that they found this rare opportunity for open exchange especially valuable.

CONCLUSIONS TO BE DRAWN FROM THE EVALUATION

As evaluators our task is to uncover what is behind this praise. Based on our data, there were a number of conclusions to be drawn:

• First, the high quality of learning materials on the platform laid the foundation.
• Second, the use of varied didactic approaches, especially the focus on interactive methods, was a central factor for engaging participants.
• Third, the VIPROM-focus on creating a safe space within training environments, which allowed for open exchange, was a key element – for students, but also for medical professionals, who e.g. felt free to discuss situations in which they felt insecure about the best course of action.
• Last, but not least, VIPROM’s focus on agency and trainers’ capability to give room to legitimate grievances but to also steer discussions to “what can be done” rather than dwelling on the multiple barriers encountered within institutions.

To summarise, the VIPROM project was an intriguing journey for us a researchers. The road to institutional change across Europe remains challenging – but it is starting, one doctor, one nurse and on midwife at a time.

MORE INFORMATION ON DOMESTIC VIOLENCE

If you are interested to learn more about domestic violence in the health sector, please visit our European training platform on domestic violence.

FOOTNOTE

1. Remark from a student in open field of the VIPROM post-training survey

ABOUT THE AUTHORS

Dr. Stefanie Mayer, is a researcher at the Institute for Conflict Studies (IKF) in Vienna. She studied political science at the University of Vienna finishing her Master in 2004. Since then she has been working as a researcher in various institutes and took part in a 3-year post-graduate programme in sociology at the Institute for Advanced Studies (IHS). 2016 she finished her dissertation on debates on racism and anti-racism in white feminist activism in Vienna. Her research focusses on gender issues in different social fields as well as critical intersectional perspectives on antifeminism and other ideologies of inequality.

Brigitte Temel, MA, is currently a researcher ath the Institute for Clonflict Studies (IKF) in Vienna. She finished her master’s degree in Gender Studies (2015) at the University of Vienna. Her research focuses on violence against women, gender and sexuality, LGBTIQ studies and antifeminism.