Are there problems with WikiTree Sourcer?

Question

I am not familiar with the WikiTree Sourcer but it seems recently that it has been responsible for creating profiles that contain errors of the type "Error 872: Named inline citation error".  I corrected these on this profile (Scarborough-3332) and then began to notice that many other profiles have the same error in the same format--several extraneous <ref> tags.  For example, other profiles are Statler-273, Dillon-6201, Witt-4578).  Looking at the Changes history of these profiles they all contain the comment "...using external data ... from fs via WikiTree Sourcer)"

Comments

Hi Barbara. Maybe the discussion at this G2G question is relevant.

---

Yes, thank you.  

It is definitely relevant but I'm having difficulty understanding the WikiTree Sourcer issue, since it's not a tool I have used.  I have been working as a Data Doctor and this week the Data Doctor Challenge of the Week is "Correct simple errors in reference tags".  We look at spreadsheets that contain links to the profiles that have reference tag errors.  The spreadsheet I was working with is https://www.softdata.si/wt/Err_20230723/868_New_0.htm  Many of the profiles with the error that I mentioned seeing in profiles like Scarborough-3332 are owned by Cynthia Crafton who asked the question in the discussion that you refer to.

It was a lot of work to find and fix the errors in Scarborough-3332 and I'm concerned about how much work would be involved in correcting all the profiles that have that same error or group of errors .  Since it appears to have come about by an automated process I was hoping there might be an automated way to correct those errors.

---

Thanks for the further details, Barbara.

Aleš Trtnik would be able to advise if an automated solution is possible. To attract his attention, you could edit your question and add the tag  ales .

It would be a good idea to leave at least one profile you find with this problem unaltered for the moment, to serve as an illustrative example.

Answer 1

This is not a problem with WikiTree Sourcer as such but is a case of a user pasting narratives with inline citations into the sources section rather than the biography section.

Sourcer can be used to fill the fields in the add person screen(s) and it can also be used to generate citations which can be pasted into the biography.

In the case of Statler-273 it looks like someone did both. They filled the data fields using WikiTree Sourcer (which is why that change history comment is there) and then they used Sourcer to "Build All Citations" on the FamilySearch profile. They had the options set to build narratives with inline citations. It looks like they must have been also using the WikiTree Browser Extension with the "Add Person Redesign" on which shows two text boxes at the bottom: "Biography" and "Sources" and they pasted the narratives with inlines citations into the Sources box which was the wrong thing to do (since inline citations should go before the == Sources == line).

If they had instead used the Sourcer option to automatically do Build All Citations then Sourcer would automatically have put the citations in the correct place. Of course it is possible that there is a bug where Sourcer would put them in the wrong place but I'm not aware of one - if someone has set of repro steps that show a bug please let me know.

Comments

If it is as I described above the fix should be pretty easy. Just move the == Sources == and <references /> lines to after the inline citations.

---

Looking at Dillon-6201 it looks like the WBE AutoBio feature was used after creating the profile. This may be what caused the mismatched refs - as discussed in the G2G question that Jim linked to. This makes it a lot harder to fix up manually. I think Ian may have fixed that issue (where AutoBio did not expect inline references after the == Sources == line and thus did not handle them correctly).

In this case there was really no reason to run AutoBio since there was already a perfectly good biography (or would have been if the narratives and inline citations were above the Sources line).

---

Rob, as a general question, with multiple extensions each with a profusion of options, are interactions (like this one apparently is) where the extensions interfere with one another going to be an increasing problem? What systematic steps can be taken to avoid this?

---

The simple solution is that everyone should either do a preview of a profile before saving it or look at profile after saving it to make sure it looks correct. Unfortunately during the thon, people are trying to create as many profiles as possible.  

In the G2G about July 2023 Highlights, I had noted "the largest group of suggestions generated from the profiles created during the thon were 1,580 profiles that have inline citations added after the references line in the Sources section." That did not include suggestions for profiles created during the last day of the thon.  The browser extensions are tools that can be used, but the person creating the profile is responsible for what the profile looks like. 

---

Barbara, you mentioned that it took a long time to fix profiles like this. For these cases where inline citations were put in the wrong place and then AutoBio was run I think the quickest solution may be to restore the data to the first item in the change history and then move the Sources/references lines to the end of the bio (after the inline references).

The downside is that you would lose later edits to the data fields if any of these were not just automated changes made by AutoBio. So those might have to be redone.

If you look at the example of Witt-4578 there are 3 changes in total, the creation, the addition of a child and a 3rd change where Auto Bio was run. So in this case it could be restored to the 2nd change (adding a child) and then the sources/references lines moved.

As an aside: I wish there was a way in the Changes tab to copy the bio text from a particular change (without all the + signs and - signs etc), then, rather than doing a restore you could just copy the bio text from the original creation and replace the current bio text with that. Maybe something for the WikiTree Browser Extension in the future.

---

Jim, to answer your question, I don't think this is a case where the extensions actually interfere with each other.

In general, it is as Linda said, the extensions are tools that allow things to be done quickly. If they are used incorrectly they therefore allow mistakes to be created quickly. In this case 4 powerful/advanced extension features were used (Add Person Redesign, Save Person Data/Set fields from..., Build all Citations, AutoBio).

I think in this particular case there may have been a messaging problem (maybe via word of mouth) when the WikiTree BEE "Get All Citations" was replaced with the Sourcer "Build All Citations". If a WikiTreer had developed a flow using Add Person Redesign, Save Person Data/Set fields from..., Get All Citations, AutoBio that was working fine and they heard that the BEE Get All Citations had been replaced with Sourcer Build All Citations and they didn't check out the options for Build All Citations but just kept using the flow without checking the results then then this is what would have happened.

Build All Citations is a superset of Get All Citations and it can be used pretty much as a straight replacement but only if the option "Add/Merge -> FamilySearch Build All Citations -> Type of citations to generate" is changed from the default of "Narrative plus Sourcer style inline citation" to either "Sourcer style source citation" or "Source citation using text on FS Sources page".

Edit: actually, I think the BEE Get All Citations also had an option to generate inline citations but that may not have been the default.

---

From looking at Dillon-6201 it appears that there are <ref> where the ending </ref> is missing the / such as <ref name='_3'>In the 1900 census Wilson (age 6) was the son of William H Dillon in Tarpon Springs, Hillsborough, Florida, United States.<ref>

The BioCheck did not pick this up. The next time I have a chance to look at it, I'll look try to see why. It should have been a ref without an ending ref, and I should check for a ref embedded inside a ref.

---

Thank you for the reply, Rob. It does seem to indicate how complex the extensions and their options are becoming. There was a recent different instance, with WBE, where complicated options caused difficulty. I hope the complexity can be kept within bounds so it doesn't become a frequent source of grief.