Suggestions News and Updates (February 18th 2024) [closed]

Question

News

• I changed the list of hints on the right side on Challenge tracker page. The hints defined by the challenge now show up when you check the challenge check box. You can see what I am talking about here: https://wikitree.sdms.si/function/WTChallenge/Challenge.htm?ProfileID=10816946 The 2 challenges with defined items are Integrators Challenge and Weevils Squashed 2024.
• Added WikiData ID to wikidata index. You can now search for wikidata=q23 to find profile(s) linked from that page.
• Added link to ML/WL addition history to Space:DBE_Project_Status reports.
• Added a possibility to search for changed profiles by a user in last month. Index name is changesmonth and you use it like changesmonth=Trtnik-2 or changesmonth="WikiTree-11" https://plus.wikitree.com/default.htm?report=srch1&Query=changesmonth%3DWikiTree-11
• Added support for json on category navigation requests. https://plus.wikitree.com/function/WTCatNavigate/Category.json?Include2=needs&Category=Personal_Categories&Levels=3&format=json
• Wrote some examples on how ti use the sql="..." conditions in search queries. See: Help:WikiTree Plus#SQL
• On Wednesday there was a new session of Ask Aleš. If you missed it, see: https://www.wikitree.com/g2g/1704673/ask-ales-february-2024

Previous News

• Based on Notables project request I added a possibility to track the date of the profile addition to the Managed list or Watch list (similar to the category history). To see it for your project, just change the WikiTreeID at the end of the URL. This link is for notables project. https://plus.wikitree.com/function/WTWatchListTime/WatchListTime.htm?Account=WikiTree-11 I started to track the dates in January. For profiles added before January 2024 they are grouped by the month of the profiles creation date. Later additions are listed at the beginning grouped by day and list type. Unlisted profiles are named Unlisted but a link to them works. If you have the rights to see the profile, you will. Profiles added after the last database dump are also shown as unlisted, but will change to the actual WikiTreeID next week. If a profile is removed from the list and later readded, it will show as new addition. If a profile is merged into another or LNAB is changed, it will also show as new addition.
• To find all profiles created during the thon you can search for profilelist="ConnectAThon\202401" on WT+ https://plus.wikitree.com/default.htm?report=srch1&Query=profilelist%3D%22ConnectAThon%5C202401%22 You can combine that with your WikiTreeID to get only the profiles you manage. If you orphaned the profiles, you can use Creator_Trtnik-2 (Replace with your wikitreeID) to get those profiles. And from results you can click Suggestions in the heading to get suggestions for them. There is also a Challenge for Connect-a-Thon Suggestions this week.
• I changed the Norfolk Island, Australia to be part of the Australia so it no longer represent an unknownregion. See: https://plus.wikitree.com/function/WTShowTable/Table.htm?table=Countries&filter=AUS
• Added new report to WT+ to find Geodesic Cycles on a path https://plus.wikitree.com/default.htm?report=disp3 See https://www.wikitree.com/g2g/1685340/cycles-and-holes-in-the-big-tree and Space:Cycles_and_Holes_in_the_Big_Tree for more.
• On Wednesday there was a new session of Ask Aleš. You can watch it here https://www.wikitree.com/g2g/1683323/ask-ales-januar-2024
• Added Páez to ignore list for 7x5 suggestions.
• I changed the condition for 7x1 suggestions to better support unicode characters. Regex is now [^\p{L}\p{M}\p{Cf} .'-]
• Added 2 London reports for Greater London and City of London.
• Updated locations table for London. https://plus.wikitree.com/function/WTShowTable/Table.htm?table=Countries&filter=ENG
• Added support for 4 parts of location like City of London, Middlesex, England, United Kingdom. If I said in the past that 4 part locations are not possible in the table, now they are and remind me of the locations I can add.
• Added validation of parameters for England Sticker.
• Updated template parameter usage reports for parameters with lists. Items that are in the list are bold. For example see: https://www.softdata.si/wt/Templ_20231126/Template_World_War_II_Param1.htm

Challenge

https://www.wikitree.com/g2g/1705681/data-doctor-challenge-week-correct-simple-errors-reference

Answer 1

Apologies if this is premature, but the Unsourced and Unconnected Lists seem to be taking longer to run than usual this week.  I was wondering if there was possibly a glitch in the system.

(Thanks for the extra search and the SQL notes)

Comments

Unconnected reports completed 8 hours ago. There seems to be a problem with Unsourced reports. I restarted it now.

This week the database dump on SQL server failed and the restarted database dump was done 5 minutes after I went to bed. I started the weekly data processing in the morning (20 hours late).

The point when I post this G2G message is when the data processing and main report is done. I also export the suggestions to main SQL server and WikiTree+ starts to use the new data. After the publish, I start the creation of all other reports and it should take 12 more hours.

As the things stand right now, the most delay in data processing is caused by 50000 new URL links to validate each week. I start it as soon as possible but It takes almost 24 hours to validate them. I do have to leave some time between 2 requests to the same domain to prevent getting banned and not to overload their servers and some servers take a lot of time (up to a minute) to respond.

Another long process is FG, where I have to check 30000 new memorials each week. It takes over 12 hours to do it but I found a solution to precheck half of the links on Saturday and the process now completes in 7 hours. It seems I will have to do something similar to URL validator.

---

Thanks Ales - I can see all the new reports now.  As always, your help and explanations are appreciated.