Sm@rt WORK(ing)

Project

“Sm@rt Work” is an up to date project. The time is right for smart developments in the health sector. The aim of our project is to make the (work-)life of construction worker easier, safer, and healthier. But it is not our goal to force blue-collar workers to activities. Through our inputs, blue-collar workers should recognize that the proposed interventions help them to get fitter and healthier.

• Relevance of the project from a global healthcare perspective:
  • Among construction workers, there are many work-related illnesses
  • If this project is successful and contributes to lessen incidents of work related illnesses among construction workers → maybe the idea is used for other workers and could prevent more illnesses in Austria or even other countries
• Relevance of the project for employers:
  • Fewer work days lost to illness or rehabilitation
  • Higher employee loyalty
• Relevance of the project for employees:
  • Healthier in the workflow
  • Higher well-being both in private life and work life
• Quality of innovation:
  • The project aims at simple accessibility and maintainability by using digital technologies
  • Based on the answers to the questionnaire, the system automatically derives the optimal interventions

Usability test description

For the project “Sm@rt Work” the usability of a developed app was tested for usability and comprehensibility. The usability test is intended to show how customers can cope with the product in general. As a result, errors or complications can be detected that were not noticed in the development. Probands are invited to use the product under supervision and are observed during use. Central questions are whether the test persons can find all the relevant information and use the product without interruption. Errors can be detected, whether the linguistic formulations are unequivocal or whether something should better be reworded. The “Sm@rt Work” app was tested by the “User Experience Questionnaire” (UEQ). This test was created in 2005. A data analytical approach was used in order to ensure a practical relevance of the constructed scales, i.e. the scales were derived from data concerning a bigger pool of items. Each scale describes a distinct quality aspect of an interactive product. The items are scaled from -3 to +3, to rank positive and negative opinions. The UEQ contains 6 scales with 26 items:

• Attractiveness: Overall impression of the product.
  Do users like or dislike the product?
• Perspicuity: Is it easy to get familiar with the product?
  Is it easy to learn how to use the product?
• Efficiency: Can users solve their tasks without unnecessary effort?
• Dependability: Does the user feel in control of the interaction?
• Stimulation: Is it exciting and motivating to use the product?
• Novelty: Is the product innovative and creative?
  Does the product catch the interest of users?

The test for the mock-up of “Sm@rt Work” took place on November 2, 2018 in seminar room 1.10, 1.12 & 1.14  at FH St. Pölten. For further information please check our mock-up attached. 6 probands were invited.

Usability test procedure

The probands were separately invited to the supervisor in the room. They were provided with a smartphone on which the mock-up of the “Sm@rt Work” app was loaded. At the beginning the test process was personally explained and the agreement of the use of anonymous data was verified by signature.

The Test person was now left alone with the instructions of the test including the smartphone. All login data and the procedure were explained in writing on the instructions. Step by step, respondents should answer the questionnaire on their own health behavior and read the recommended actions.

Following the questionnaire, the probands should answer a short quiz on health issues.

The task of the supervisor was to observe the test and note the behavior during use (hesitation, uncertainty, etc).

During the test phase of the quiz, it was noted whether the test person answered the question correctly or incorrectly.

Duration of the test per person: approx. 5 – 10 minutes

Details of the test running

Course seminar room 1.10 & 1.12

• Probands come to the appropriate seminar room at the selected time
• Welcoming the subject by the supervisor
• Sign a declaration of agreement
• Hand over the task
• Probands performs usability test
• Supervisor observes during test
• Test leader sends test person after test end in seminar room 1.14 to fill out questionnaire

Course seminar room 1.14

• Welcoming the probands by the supervisor
• Hand over the questionnaire
• Adoption

Usability test results

Overall, we considered the test very successful. Testers were seeing this interface for the very first time and were operating under some pressure. Despite the fact that, only 3 of the 6 testers used mobile apps on their own smartphones, they all had no problems to achieve all tasks.

Age range of participants | Copyright: Bernhard Ruhrhofer, Max Jesenko, Katharina Maier, Eva Kamper, Florian Schweifer, Niklas Stockreiter, Bogdan Negrei

Gender segmentation | Copyright: Bernhard Ruhrhofer, Max Jesenko, Katharina Maier, Eva Kamper, Florian Schweifer, Niklas Stockreiter, Bogdan Negrei

Results of the usability test | Copyright: Bernhard Ruhrhofer, Max Jesenko, Katharina Maier, Eva Kamper, Florian Schweifer, Niklas Stockreiter, Bogdan Negrei

Values between -0.8 ad 0.8 represent a more or less neural evaluation of the corresponding scale,  values > 0,8 represent a positive evaluation and values < -0,8 represent a negative evaluation. The range of the scales is between -3 (horribly bad) and +3 (extremely good).

Results of the Usability Test - Scales | Copyright: Bernhard Ruhrhofer, Max Jesenko, Katharina Maier, Eva Kamper, Florian Schweifer, Niklas Stockreiter, Bogdan Negrei

Attractiveness is a pure valence dimension. Perspicuity, Efficiency and Dependability are pragmatic quality aspects (goal-directed), while Stimulation and Novelty are hedonic quality aspects (not goal-directed).

As noted above, Table 1. shows the results of the usability test for the 6 scales and Figure 4. shows the 26 items of the usability test. The attractiveness scale has 6 items, all other scales have 4 items.

The outcome of our testing is that the user get familiar with the product very easy. Moreover the users like the design and handling of the product. Last but not least, due to the high acceptance the users could solve their tasks without any unnecessary effort.

Postscript

The project “Sm@rt work” consists of 7 project team members and can be divided in three major parts: the questionnaire on the smartphone APP, the automatically responds by the APP with advices to increase health and the gamification part to get goodies from the company. The part “questionnaire” is the main part, in which everybody from the project team is included. Mr. Dr. Lukas Küster, who represents STRABAG Austria, has a veto-right for the customer. Romana Bichler is our project coach. Our customer (and financier) is STRABAG Austria.

Every project team member is responsible for a part of the questionnaire and the derived interventions. Niklas Stockreiter is responsible for psychological and mental stress. He is also the creative mind behind the presentations. Katharina Maier is a specialist for ergonomics and movements. Eva Maria Kamper researched for nutrition and drugs. Max Jesenko is the specialist for noise exposure. Florian Schweifer’s research deals with health prevention. Bernhard Ruhrhofer researches for skin neoplasms caused by UV rays. He is also the project leader and responsible for the summaries of every meeting.

In the second semester the team worked on developing the design of the APP which main part was done by Bogdan Negrei and Max Jesenko.

The main part of the app is to ask employees if they are healthy and if they are aware of what is healthy and what is unhealthy. Examples of questions are “often work overhead” or “drink enough water”. The questions are divided into groups and depending on the YES / NO answer are forwarded to the next question, so that the other questions (eg. smoker / non-smoker) apply to the respective person. For each answer, there is a praise, a tip, or an improvement suggestion to health and well-being.

Every day a different health issue is asked, it is not possible to answer all questions at once. Pictures, GIF’s and information videos complete the questions. A small game with quiz questions is also used as a motivation, so that the employees regularly deal with the APP and the questions about health. For a high number of points, there will be small goodies from the STRABAG company.

Also a short video for the final presentation was produced by Bogdan Negrei, Max Jesenko, Niklas Stockreiter and Katharina Maier.

On a final note: We are a team, and everybody works with everybody. Everybody helps everybody.