Wikidata:Usability and usefulness/Study 17-08 Workflows and Motivations

Meta

This study is finished. You can read about the results here ↓ or at the Slides on Commons.

Part of the study was a workshop at Wikidatacon 2017, where editors documented their editing workflows: Category on commons with raw data and summaries

If you are interested: Our methods are summarized here. User: Jan Dittrich (WMDE) wrote about the Workshop’s method here.

Thanks to everyone who participated and helped us!

Result summary

Executive summary

We learned that Wikidata Editors…

• …worked from lists (Petscan; Listeria, Recent changes…) which provide an overview of ToDos and provide links to the items (see graphic below)
• …had large ("reduce gender gap") and more concrete topics ("add woman painters from the 18th century from a catalog") they care about
• …used references, ranks and value specifiers rarely. These functions are often not understood as intended.

Editing Workflow

Based on Workshop-participants 3,6,8,4 and users 1,2,3,4 from interviews

The workflow for creating new items is similar. I [Jan Dittrich, WMDE] have fewer observations of it, though. The lists are different (not lists of Wikidata items) and there is obviously an additional creation step before statements are added

(Very active) Users have a special interest that motivates them

Our users often have

• a mission (e.g. make female artists more visible)
• Mission-repeated sub-tasks (e.g. link artworks to newly added female artists)

This corresponds to WMF’s finding that   “People ​ edit Wikipedia for diverse reasons, most of which serve purposes beyond [just] editing Wikipedia from the  New Editors Research  for Wikipedia

Users work from a list

Lists with links to items are a core tool for frequent editors

• Lists give an overview of To-Dos
• Lists give access to relevant items
• Lists give the opportunity to check progress

Lists can be e.g.: listeria generated, recent changes, new items, watchlist, petScan, external tracking via Excel, self-generated RSS-feed…

See Sum of all Paintings – Bring these lists down to zero for examples

Getting in external data, often using gadgets

• frequent editors use gadgets and tools to add data.
• They offer semi-automatic access and sometimes even editing
  • Authority Control
  • Wikidata useful (useful property suggestions)
  • Wikipedia Sidebar (shows related WP article)
• Edits without such tools were rare among the people I [Jan Dittrich, WMDE] observed

More of the same – Repetitive tasks

Many edits have the same structure:

• Several versions of the same image need the same “depicts” values
• On one item, the references can be the same
• Several similar items have the same structure

Currently people do:

• Modeling items according to an example item. If they want to model a statue, they look up Michaelangelo’s David and use the same properties
• Modeling items using a description from a Wikiproject
• Use an often defunct “copy reference”-Gadget
• Repeat the same task again and again manually.

Mass imports using a complex toolchain

Mass imports are done with an elaborated, bricolage-y toolchain, combining Excel, Python-Scripts, QuickStatements and non-obvious quality assurance steps

References, Values, Ranks: rarely used, not understood

• References are added when statements can be controversial (date of death)
• They are also added when they are easy to reach in general

Snak types and ranks are poorly understood. The knowledge seems to be based on examples for use, not on their intended abstract meaning. (“You use ‘preferred’ if a better scientific result comes out”*, “No value if the person’s term has not ended yet”*)

Recommendations

Explore Working from Lists

Working from lists seems to be a common practice – can we make this easier and better integrated?

Ease work with repetitive edits

Creating or editing items is often repetitive. There are several possibilities to make this easier for manual edits e.g. copy/paste of items, statements and references.

Also, the infrastructure for bot- and-gadget edits could be improved.

Draft a strategy for tools and userscripts

Userscripts, gadgets and tools are core part of the editing experience. Nevertheless, they are build on an infrastructure that causes the organizations WMF/WMDE and the users repeated problems:

• It is hard to find the “right” gadgets/userscript/tool for one’s work
• Gadgets often break the Wikidatas UI or Wikidatas UI breaks the Gadgets
• Gadgets/userscript/tools have often an just OK usability.

It seems to be an open question of how we should deal with that tension (high importance, difficult UI, not integrated infrastructure)

Research and actions on references, value specifiers and ranks

There seem to be problems of understanding and using references, value specifiers and ranks.

• Quantitatively assess ranks’ and value specifiers’ actual use. Partly we could do this against the initial assumptions made when creating the data model
• Quantitatively assess the use of references in terms of correlation with specific properties, manual vs. bot creation, quality of references.

Related Works

• Lists in the Wikimedia movement? Why? What? (November 21, 2019 by Astinson-WMF)

Retro

• The co-documentation workshop went well: People seemed to be able to participate without much trouble, the information generated was useful for UX (and as well, we hope, for the community directly to see each others workflows)
• Our participants are again among the very active ones. We tried to expand the call-for-participants a bit, using not only personal referral and mailing lists, but also social media. However, we should see if we can get users for whom Wikidata is less a part of their "volunteer identity" but a mere tool.

Original Proposal/Call for participants

Goal:

We want to find out how and why people edit on wikidata, including new and semi-experienced people.

Knowing more about that helps us to improve the user interface of Wikidata and Wikidata’s functionality.

The research would be done by Jan, who is UX Designer and Researcher for the Wikidata team.

That seems to be at least a bit familiar? Great – you can help us to understand Wikidata use better!

Approach:

We would like to learn from people who edit wikidata about their work on wikidata and how they do it.

Usually, we do this via a

• hangouts
• skype
• open source meet.jitsi.

To understand what you do, it can be useful to share your screen.

If you are in-depth interested in our method, have a look at Jan’s book on learning about user activities and motivations

What would we talk about:

Here some examples of what would talk about:

• What are the items you add about and how do you add them?
• How do you find problems with the data and how do you fix them?
• What is annoying for you now?

Who can participate:

Everyone who edited on Wikidata.org already and does add and edit information via the Wikidata main interface. We try to get people of different experience levels and in practice we often lack people who only edit from time to time. If you are one of these: You could diversify our sample a lot!

If you want to participate:

Write on Jan’s talkpage or send him a mail to jan.dittrich AT wikimedia.de