Export all books as TSV stalls at 0 books

DiscussieBug Collectors

Sluit je aan bij LibraryThing om te posten.

Export all books as TSV stalls at 0 books

1bnielsen
jun 3, 2021, 11:35 am

I'm trying to export my books as TSV but it seems stuck on 0 books :-(
Normally it exports without any trouble, so something broke yesterday, methinks.

2kristilabrie
jun 3, 2021, 12:09 pm

Thanks for the report, checking.

3kristilabrie
jun 3, 2021, 12:32 pm

Okay, ccatalfo thinks he has this fixed, if you want to try again?

4bnielsen
Bewerkt: jun 3, 2021, 1:44 pm

0 books, 100 books, 200 books, yes it's working much better now.
I'll report when it has finished, but so far it seems like it's fixed!
Thanks!
ETA: all books exported as TSV without a hitch. I'm closing this bug report.

5frebbe
Bewerkt: jun 4, 2021, 12:32 pm

Sorry to say, this error still occurs here (Google Chrome Version 91.0.4472.77) when I try to export my ~ 250 books to whatever format. I see the rotating wheel "Bücher verarbeitet" for hours without any output. Also tried with latest versions of MS Edge and Mozilla Firefox. The same. There is absolutely no progress in exporting. I do not receive a file with zero content, I can't get any output. Please let me know, what I'm doing wrong or how to solve the problem. Thank you very much.

6kristilabrie
jun 4, 2021, 1:04 pm

>5 frebbe: Hi frebbe, both ccatalfo (one of our developers) and myself are able to export on the .de site for your account. I presume you're using librarything.de, is that correct?

Do you mind clearing your browser's cache/cookies and trying the export once more? If they're still failing for you, please let me know which export/s you're trying and also perhaps your current IP address: you can find your IP by going to https://www.whatsmyip.org and finding the series of numbers and periods at the top of the page. We can take a closer look with that. Thanks.

7frebbe
jun 4, 2021, 3:52 pm

>6 kristilabrie: I was using https://www.librarything.de/export.php?export_type=xls and - following your advice - was able to download the librarything_frebbe.xls file. Thank you very much for your help!

8kristilabrie
jun 5, 2021, 5:44 pm

Excellent! Glad to hear it.

9Lisa2013
jun 7, 2021, 3:43 pm

I still can't export books. Stalls at 0 books expoeted. For Excel, for Tab-delimited text too.

On a Mac in Firefox 89.0 up to date. Haven't tried other browsers yet but this is repeatable. Several tries so far.

Thanks.

10bnielsen
jun 8, 2021, 1:04 am

>9 Lisa2013: I get the same behaviour. No errors, just stuck at 0 books. I've reopened this bug report and messaged ccatalfo

11bnielsen
jun 8, 2021, 8:00 am

Hmm, just tried again and this time it worked perfectly. Did anyone fix it?

12ccatalfo
jun 8, 2021, 9:23 am

>9 Lisa2013: >10 bnielsen: Sorry folks, still working out what is *sometimes* causing a memory issue.

>9 Lisa2013: Can you please try once again and we'll see if this next code change gets it through? If not, I will up it once more and we can see where threshold needs to be.

Sorry for the inconvenience.

13ccatalfo
jun 8, 2021, 10:04 am

>10 bnielsen: When your export failed but later succeeded - was it the very same collection or set of books you were exporting?

14bnielsen
Bewerkt: jun 8, 2021, 2:23 pm

>13 ccatalfo: The same. But I tried various versions in between to see if I could interrupt the "0 books" stall. But I didn't succeed in anything until suddenly the full export worked again.

15ccatalfo
jun 8, 2021, 10:29 am

>14 bnielsen: Thanks. How peculiar. Digging deeper.

16ccatalfo
jun 8, 2021, 10:32 am

>14 bnielsen: Aha. The consumer process is eating up memory like it's candy, until it's eaten it all. And thus the eventual memory problem.

I should have checked that sooner, but your before and after example lead me to that. Thanks!

Now, to figure out why.

17ulmannc
jun 8, 2021, 10:56 am

I tried running the export again and I'm stalled again. It worked after it worked as described in >11 bnielsen:. Now nothing. All I have right now is an Android v 6 phone with the current vesion of Google Chrome. Will not get back to a pc until tonight.

18ccatalfo
jun 9, 2021, 7:15 am

>17 ulmannc: Ok I've made some more adjustments - principally attempting to get rid of an apparent memory leak in the excel library we use.

How are things now? I do not see the same error rate as before these changes, so I am hopeful this will fix it completely...

19ulmannc
jun 9, 2021, 9:44 am

>18 ccatalfo: I just tried it. It's been running for about 5 minutes and it is sitting on zero. I'm running Windows 10 Pro with the current version of Google Chrome.

Just for ha ha's, I also tried an export with a tag filter of "Charles Marion Russell" and got nothing as well. The count on those tags is 185

20bnielsen
jun 9, 2021, 10:01 am

It worked earlier in the day, but now I'm back to the perpetual "0 books processed".

21bnielsen
jun 9, 2021, 2:10 pm

And now it worked again. Seems a bit wonky :-)

22kristilabrie
jun 9, 2021, 2:13 pm

Yes, ccatalfo added in some improvements that should (fingers crossed) fix these export issues we've been having. Please post here if you're still seeing trouble! Thanks for your patience, all.

23bnielsen
jun 9, 2021, 2:37 pm

Currently stalled at the perpetual "0 books processed".

24LibraryDraytonHall
jun 9, 2021, 5:50 pm

Hello - haven't gotten Export (to EXCEL) to work at all today. Tried clearing cache (using Chrome) and still, no go...is there something else I should be trying or is Export "broken"....let Export run a few minutes (on a library of less than 800 books) but stuck at "0 books processed". Please fix. I love this app and need to generate a spreadsheet! Thanks very much for your attention to this matter.

25bnielsen
jun 10, 2021, 2:33 am

Currently stalled at the perpetual "0 books processed". (That's after a system update and reboot of my machine, so I don't think anything browser related survived that :-)

My latest succesful export was >21 bnielsen:.

26ccatalfo
jun 10, 2021, 8:13 am

Ok once again thanks for your patience - I'm tweaking the logic around it and figuring out how quickly we can process them. A bit of a queue was worked up overnight, unfortunately. I've cleared it this morning and am watching to see how it behaves.

27bnielsen
jun 10, 2021, 10:19 am

Just exported my books as TSV and everything looks fine.

28ulmannc
jun 10, 2021, 10:37 am

Just tried it again and after 5 min it is sitting at zero. The number normally starts going after about 10 sec

29kristilabrie
jun 10, 2021, 11:17 am

>28 ulmannc: Noted, passing to ccatalfo!

30ccatalfo
jun 10, 2021, 1:02 pm

Yes, ok, we've made some more changes to help XLS export along - I'm not seeing failures in the logs right now, even with larger libraries.

We're also looking at the speed (of XLS in particular) to see if we can improve that.

31LibraryDraytonHall
jun 10, 2021, 4:26 pm

Success! I was just able to export all my books (>700) via Excel. (I come from a software development background and really appreciate the prompt support!) Thanks, again!

32ulmannc
jun 10, 2021, 7:32 pm

Just ran a full tab delimited export (approx 8400 entries) and it worked. Took a bit under 10 minutes which was about where it was before the "excitement" . I appreciate your persistence on this!

"The Electronic Luddite"

33fjakhdf
jun 10, 2021, 9:03 pm

Works fine now, thanks!

34ccatalfo
jun 11, 2021, 7:04 am

Thanks, all, this is good to hear.

I see no errors showing up since these changs and the queue appears nice and empty.

Please report back if you encounter problems.

Meanwhile we'll see if we can speed it up a bit.

35kristilabrie
jun 24, 2021, 10:05 am

Marking as fixed.