Wikidata:Requests for permissions/Bot/DuraEuroposBot
The following discussion is closed. Please do not modify it. Subsequent comments should be made in a new section. A summary of the conclusions reached follows.
Approved--Ymblanter (talk) 18:19, 11 October 2020 (UTC)
DuraEuroposBot[edit]
DuraEuroposBot (talk • contribs • new items • new lexemes • SUL • Block log • User rights log • User rights • xtools)
Operator: Valeriummaximum (talk • contribs • logs)
Task/s: This bot will be used to add dates to a number of archaeological excavated at Dura-Europos (Q464266) and held at Yale University Art Gallery (Q1568434), approximately 15,000. We plan to use Quickstatements and OpenRefine for a significant amount of the batch-upload but face some issues with these tools when it comes to dates, so we will write a script for adding dates and qualifiers for inception (P571). From our experience with Quickstatements at the moment, we cannot add qualifiers to "somevalue" with "inception" nor specify calendar. OpenRefine also cannot deal with BCE and auto-converts Julian time to Gregorian.
Code: We will use Pywikibot. Essentially, when an object has a date range like 113BCE-256CE, we will parse this out as inception (P571): claim.setSnakType('somevalue'), with qualifiers earliest date (P1319) -0113-00-00T00:00:00Z/9, latest date (P1326)+0256-00-00T00:00:00Z/9. We will also add statements to specify Julian and Gregorian calendar when necessary. Batchupload will also give unique labels and IDs, collection, location of excavation, with references for each statement to Yale Gallery website for the item.
Draft code is [here]
To avoid duplicates, the code will embed SPARQL query, compare unique YUAG ID for Yale Gallery objects in Wikidata and IDs in our data, and then compare statements. This bot will also allow us to update labels and descriptions to uniquely identify each object with its ID and inventory number.
Function details: create item pages for Yale Dura Europos collection, add labels, add Yale University Art Gallery ID (P8583), add dates with qualifiers, add links to public domain images Commons compatible image available at URL (P4765). --Valeriummaximum (talk) 09:44, 29 September 2020 (UTC)
Discussion[edit]
Hello, I've been testing parts of this code in Wikidata and test.Wikidata and the code should be close to functional. I am wondering if I could get permission to run a few test edits using the bot on a small slice of our data to see if it works. (the process for applying for botflags in new to me, so please any advice or suggestions would be welcome) Valeriummaximum (talk) 12:15, 5 October 2020 (UTC)
- Yes, please make about 50 test edits.--Ymblanter (talk) 17:33, 6 October 2020 (UTC)
- Many thanks. Just to report back, I uploaded the data for fifty items in our collection. At the initial stage, there was a bug in the code that caused a few API errors and created a few empty or duplicate entries. I cleaned those up and reran the code, and the code was able to iterate over existing items and not duplicate. One error I noticed is that, when updating items to add "instance of archaeological artifact", I added a reference; but when creating a new item, I did not add a reference. I can fix this in the code and in situ. Here is a list of 50 items that were created/updated: Q49303880, Q100156231, Q100156248, Q100165906, Q100165679, Q100165966, Q100166008, Q100166009, Q100166015, Q100166020, Q100191326, Q100191746, Q100192143, Q100192534, Q100192936, Q100193303, Q100193683, Q100194084, Q100194477, Q100194857, Q100195241, Q100195598, Q100195938, Q100196247, Q100196594, Q100196940, Q100197315, Q100197689, Q100198066, Q100198447, Q100198817, Q100199144, Q100199473, Q100199830, Q100200238, Q100200635, Q100166283, Q100166292, Q100201256, Q100201735, Q100202213, Q100202690, Q100203189, Q49303659, Q49303712, Q49303767, Q100204083, Q100204442, Q100204776, Q100205112 Valeriummaximum (talk) 18:29, 8 October 2020 (UTC)
- Looks good to me, I can approved the bot in a couple of days provided no objections have been raised.--Ymblanter (talk) 19:30, 9 October 2020 (UTC)
- Many thanks @Ymblanter:. I just have one question (I hope not a silly one), is there a throttle policy on bots interacting with Wikidata? Currently the PAWS server defaults to a 9 second delay between edits but I am wondering if I am allowed to speed this up (without violating the maxlag parameter).Valeriummaximum (talk) 21:54, 9 October 2020 (UTC)
- Have a look here: Wikidata:Project chat#Remove noratelimit for bots--Ymblanter (talk) 12:02, 10 October 2020 (UTC)
- Many thanks @Ymblanter:. I just have one question (I hope not a silly one), is there a throttle policy on bots interacting with Wikidata? Currently the PAWS server defaults to a 9 second delay between edits but I am wondering if I am allowed to speed this up (without violating the maxlag parameter).Valeriummaximum (talk) 21:54, 9 October 2020 (UTC)
- Looks good to me, I can approved the bot in a couple of days provided no objections have been raised.--Ymblanter (talk) 19:30, 9 October 2020 (UTC)
- Many thanks. Just to report back, I uploaded the data for fifty items in our collection. At the initial stage, there was a bug in the code that caused a few API errors and created a few empty or duplicate entries. I cleaned those up and reran the code, and the code was able to iterate over existing items and not duplicate. One error I noticed is that, when updating items to add "instance of archaeological artifact", I added a reference; but when creating a new item, I did not add a reference. I can fix this in the code and in situ. Here is a list of 50 items that were created/updated: Q49303880, Q100156231, Q100156248, Q100165906, Q100165679, Q100165966, Q100166008, Q100166009, Q100166015, Q100166020, Q100191326, Q100191746, Q100192143, Q100192534, Q100192936, Q100193303, Q100193683, Q100194084, Q100194477, Q100194857, Q100195241, Q100195598, Q100195938, Q100196247, Q100196594, Q100196940, Q100197315, Q100197689, Q100198066, Q100198447, Q100198817, Q100199144, Q100199473, Q100199830, Q100200238, Q100200635, Q100166283, Q100166292, Q100201256, Q100201735, Q100202213, Q100202690, Q100203189, Q49303659, Q49303712, Q49303767, Q100204083, Q100204442, Q100204776, Q100205112 Valeriummaximum (talk) 18:29, 8 October 2020 (UTC)