sql/stats/2022-05-11_crawl_changelog.txt


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410

                             domain                              |         status          | count  
-----------------------------------------------------------------+-------------------------+--------
 academic.oup.com                                                |                         |   2210
 academic.oup.com                                                | no-pdf-link             |   1350
 academic.oup.com                                                | bad-redirect            |    510
 academiccommons.columbia.edu                                    |                         |    379
 academiccommons.columbia.edu                                    | success                 |    339
 aip.scitation.org                                               |                         |    762
 aip.scitation.org                                               | terminal-bad-status     |    430
 apps.crossref.org                                               |                         |   9894
 apps.crossref.org                                               | no-pdf-link             |   9886
 apps.euskadi.eus                                                |                         |    242
 apps.euskadi.eus                                                | no-pdf-link             |    240
 arxiv.org                                                       |                         |  44889
 arxiv.org                                                       | success                 |  28781
 arxiv.org                                                       | spn2-backoff            |   7975
 arxiv.org                                                       | terminal-bad-status     |   4508
 arxiv.org                                                       | spn2-cdx-lookup-failure |   2010
 arxiv.org                                                       | redirect-loop           |    619
 arxiv.org                                                       | no-pdf-link             |    242
 arxiv.org                                                       | spn2-error              |    236
 asa.scitation.org                                               |                         |    356
 asa.scitation.org                                               | terminal-bad-status     |    299
 asmedigitalcollection.asme.org                                  |                         |    240
 assets.cureus.com                                               |                         |    336
 assets.cureus.com                                               | success                 |    335
 assets.researchsquare.com                                       |                         |   1042
 assets.researchsquare.com                                       | success                 |    993
 av.tib.eu                                                       |                         |    205
 av.tib.eu                                                       | no-pdf-link             |    203
 bibliographie.uni-tuebingen.de                                  |                         |    213
 bibliographie.uni-tuebingen.de                                  | no-pdf-link             |    211
 biorxiv.org                                                     | redirect-loop           |    217
 biorxiv.org                                                     |                         |    217
 books.openedition.org                                           |                         |    691
 books.openedition.org                                           | no-pdf-link             |    687
 boris.unibe.ch                                                  |                         |    525
 boris.unibe.ch                                                  | success                 |    466
 bridges.monash.edu                                              |                         |    663
 bridges.monash.edu                                              | no-pdf-link             |    647
 brill.com                                                       |                         |    860
 brill.com                                                       | success                 |    434
 chemrxiv.org                                                    |                         |    201
 classiques-garnier.com                                          |                         |    242
 content.iospress.com                                            |                         |    325
 content.iospress.com                                            | link-loop               |    247
 core.tdar.org                                                   |                         |    216
 core.tdar.org                                                   | no-pdf-link             |    211
 cyberleninka.ru                                                 |                         |    646
 cyberleninka.ru                                                 | success                 |    620
 d197for5662m48.cloudfront.net                                   |                         |    263
 d197for5662m48.cloudfront.net                                   | success                 |    262
 dergipark.org.tr                                                |                         |    891
 dergipark.org.tr                                                | success                 |    526
 dergipark.org.tr                                                | no-pdf-link             |    261
 digi.ub.uni-heidelberg.de                                       |                         |    427
 digi.ub.uni-heidelberg.de                                       | no-pdf-link             |    427
 direct.mit.edu                                                  |                         |    268
 direct.mit.edu                                                  | no-pdf-link             |    208
 dl.acm.org                                                      |                         |   1719
 dl.acm.org                                                      | success                 |    829
 dl.acm.org                                                      | no-pdf-link             |    546
 dl.acm.org                                                      | terminal-bad-status     |    205
 dlc.library.columbia.edu                                        |                         |    385
 dlc.library.columbia.edu                                        | terminal-bad-status     |    319
 doi.ala.org.au                                                  |                         |    724
 doi.ala.org.au                                                  | no-pdf-link             |    721
 doi.apa.org                                                     |                         |    214
 doi.org                                                         |                         |   3390
 doi.org                                                         | terminal-bad-status     |   2938
 doi.org                                                         | redirect-loop           |    233
 doi.org                                                         | spn2-wayback-error      |    208
 doi.usp.org                                                     |                         |    325
 doi.usp.org                                                     | no-pdf-link             |    324
 downloads.hindawi.com                                           |                         |   1439
 downloads.hindawi.com                                           | success                 |   1436
 du.diva-portal.org                                              |                         |    589
 du.diva-portal.org                                              | success                 |    586
 econtents.bc.unicamp.br                                         |                         |    310
 econtents.bc.unicamp.br                                         | success                 |    310
 ediss.uni-goettingen.de                                         |                         |    728
 ediss.uni-goettingen.de                                         | success                 |    425
 elibrary.kdpu.edu.ua                                            |                         |    907
 elibrary.kdpu.edu.ua                                            | bad-redirect            |    712
 elibrary.ru                                                     |                         |    925
 elibrary.ru                                                     | terminal-bad-status     |    492
 elibrary.ru                                                     | bad-redirect            |    230
 elibrary.vdi-verlag.de                                          |                         |    393
 elifesciences.org                                               |                         |    296
 elifesciences.org                                               | success                 |    276
 europepmc.org                                                   |                         |   3024
 europepmc.org                                                   | success                 |   2541
 europepmc.org                                                   | terminal-bad-status     |    463
 figshare.com                                                    |                         |    493
 figshare.com                                                    | no-pdf-link             |    440
 files.osf.io                                                    |                         |    883
 files.osf.io                                                    | success                 |    686
 fjfsdata01prod.blob.core.windows.net                            |                         |   3869
 fjfsdata01prod.blob.core.windows.net                            | success                 |   3818
 ieeexplore.ieee.org                                             |                         |  10854
 ieeexplore.ieee.org                                             | gateway-timeout         |   5495
 ieeexplore.ieee.org                                             | spn2-backoff            |   1662
 ieeexplore.ieee.org                                             | no-pdf-link             |   1417
 ieeexplore.ieee.org                                             | success                 |   1410
 ieeexplore.ieee.org                                             | redirect-loop           |    768
 iiif.crossasia.org                                              |                         |   7608
 iiif.crossasia.org                                              | no-pdf-link             |   7568
 ikee.lib.auth.gr                                                |                         |    450
 ikee.lib.auth.gr                                                | success                 |    332
 ins.journals.ekb.eg                                             |                         |    212
 iopscience.iop.org                                              |                         |    268
 jamanetwork.com                                                 |                         |    333
 journals.aps.org                                                |                         |    414
 journals.asm.org                                                |                         |    242
 journals.flvc.org                                               |                         |    245
 journals.flvc.org                                               | success                 |    242
 journals.healio.com                                             |                         |    755
 journals.healio.com                                             | terminal-bad-status     |    668
 journals.lincoln.ac.nz                                          |                         |    244
 journals.lincoln.ac.nz                                          | success                 |    239
 journals.lww.com                                                |                         |   1772
 journals.lww.com                                                | link-loop               |   1425
 journals.lww.com                                                | spn2-backoff            |    209
 journals.openedition.org                                        |                         |   1192
 journals.openedition.org                                        | redirect-loop           |    467
 journals.openedition.org                                        | success                 |    451
 journals.plos.org                                               |                         |    771
 journals.plos.org                                               | success                 |    750
 journals.ub.uni-heidelberg.de                                   |                         |    787
 journals.ub.uni-heidelberg.de                                   | success                 |    741
 kazanmedjournal.ru                                              |                         |    240
 kazanmedjournal.ru                                              | success                 |    231
 kiss.kstudy.com                                                 |                         |    219
 kiss.kstudy.com                                                 | no-pdf-link             |    218
 kluwerlawonline.com                                             |                         |    444
 kluwerlawonline.com                                             | link-loop               |    402
 libraetd.lib.virginia.edu                                       |                         |    362
 libraetd.lib.virginia.edu                                       | no-pdf-link             |    361
 link.springer.com                                               |                         |    305
 linkinghub.elsevier.com                                         |                         |    568
 linkinghub.elsevier.com                                         | spn2-backoff            |    545
 ltu-figshare-repo.s3.aarnet.edu.au                              |                         |    269
 ltu-figshare-repo.s3.aarnet.edu.au                              | success                 |    268
 mausamjournal.imd.gov.in                                        |                         |    202
 mdpi-res.com                                                    |                         |   8892
 mdpi-res.com                                                    | success                 |   8863
 mededpublish.org                                                |                         |   1900
 mededpublish.org                                                | no-pdf-link             |   1900
 meetingorganizer.copernicus.org                                 |                         |    276
 meetingorganizer.copernicus.org                                 | no-pdf-link             |    271
 muse.jhu.edu                                                    |                         |   1047
 muse.jhu.edu                                                    | terminal-bad-status     |    755
 muse.jhu.edu                                                    | link-loop               |    203
 online.ucpress.edu                                              |                         |    358
 online.ucpress.edu                                              | link-loop               |    212
 onlinelibrary.wiley.com                                         |                         |   5813
 onlinelibrary.wiley.com                                         | terminal-bad-status     |   4587
 onlinelibrary.wiley.com                                         | spn2-wayback-error      |    614
 onlinelibrary.wiley.com                                         | blocked-cookie          |    381
 open.library.ubc.ca                                             |                         |    206
 opendata.uni-halle.de                                           |                         |   1768
 opendata.uni-halle.de                                           | success                 |   1215
 opendata.uni-halle.de                                           | wrong-mimetype          |    260
 opendata2.uni-halle.de                                          |                         |    206
 opg.optica.org                                                  |                         |    205
 osf.io                                                          |                         |   2949
 osf.io                                                          | no-pdf-link             |   2404
 osf.io                                                          | spn2-backoff            |    299
 papers.ssrn.com                                                 |                         |   3962
 papers.ssrn.com                                                 | link-loop               |   3800
 peerj.com                                                       |                         |    273
 preprints.jmir.org                                              |                         |    275
 preprints.jmir.org                                              | cdx-error               |    255
 publikationen.bibliothek.kit.edu                                |                         |    213
 publons.com                                                     |                         |    593
 publons.com                                                     | no-pdf-link             |    590
 pubs.acs.org                                                    |                         |   2288
 pubs.acs.org                                                    | terminal-bad-status     |   1841
 pubs.acs.org                                                    | spn2-wayback-error      |    210
 pubs.rsc.org                                                    |                         |   1698
 pubs.rsc.org                                                    | bad-redirect            |    811
 pubs.rsc.org                                                    | link-loop               |    352
 pubs.rsc.org                                                    | success                 |    307
 radiopaedia.org                                                 |                         |    220
 read.dukeupress.edu                                             |                         |    303
 repositories.lib.utexas.edu                                     |                         |   1570
 repositories.lib.utexas.edu                                     | bad-redirect            |    513
 repositories.lib.utexas.edu                                     | spn2-backoff            |    383
 repositories.lib.utexas.edu                                     | gateway-timeout         |    379
 repositories.lib.utexas.edu                                     | terminal-bad-status     |    282
 repository.uj.ac.za                                             |                         |    489
 repository.uj.ac.za                                             | no-pdf-link             |    365
 repository.unsworks.unsw.edu.au                                 |                         |    397
 repository.urosario.edu.co                                      |                         |   2429
 repository.urosario.edu.co                                      | success                 |   1648
 repository.urosario.edu.co                                      | bad-redirect            |    613
 rex.libraries.wsu.edu                                           | no-pdf-link             |    241
 rex.libraries.wsu.edu                                           |                         |    241
 rsdjournal.org                                                  |                         |    208
 rsdjournal.org                                                  | success                 |    208
 s3-ap-southeast-2.amazonaws.com                                 |                         |    282
 s3-ap-southeast-2.amazonaws.com                                 | success                 |    277
 s3-eu-west-1.amazonaws.com                                      |                         |   4615
 s3-eu-west-1.amazonaws.com                                      | success                 |   4593
 s3-euw1-ap-pe-df-pch-content-store-p.s3.eu-west-1.amazonaws.com |                         |    240
 s3-euw1-ap-pe-df-pch-content-store-p.s3.eu-west-1.amazonaws.com | success                 |    237
 sage.figshare.com                                               |                         |    415
 sage.figshare.com                                               | no-pdf-link             |    385
 scholar.dkyobobook.co.kr                                        |                         |    512
 scholar.dkyobobook.co.kr                                        | no-pdf-link             |    509
 scholarlypublishingcollective.org                               |                         |    287
 scholarworks.gsu.edu                                            |                         |   1132
 scholarworks.gsu.edu                                            | success                 |   1000
 scholarworks.iupui.edu                                          |                         |    205
 scholarworks.umass.edu                                          |                         |    417
 scholarworks.umass.edu                                          | success                 |    400
 sciencescholar.us                                               |                         |    404
 secure.jbs.elsevierhealth.com                                   |                         |    727
 secure.jbs.elsevierhealth.com                                   | terminal-bad-status     |    722
 tandf.figshare.com                                              |                         |    354
 tandf.figshare.com                                              | no-pdf-link             |    342
 unsworks.unsw.edu.au                                            |                         |    408
 unsworks.unsw.edu.au                                            | spn2-cdx-lookup-failure |    342
 valep.vc.univie.ac.at                                           | no-pdf-link             |    737
 valep.vc.univie.ac.at                                           |                         |    737
 watermark.silverchair.com                                       |                         |   1604
 watermark.silverchair.com                                       | success                 |   1598
 wayf.switch.ch                                                  |                         |    215
 wayf.switch.ch                                                  | no-pdf-link             |    213
 www.ahajournals.org                                             |                         |    438
 www.ahajournals.org                                             | no-pdf-link             |    306
 www.ahbps.org                                                   |                         |    316
 www.ahbps.org                                                   | success                 |    312
 www.atenaeditora.com.br                                         |                         |    390
 www.atenaeditora.com.br                                         | terminal-bad-status     |    333
 www.atlantis-press.com                                          |                         |    914
 www.atlantis-press.com                                          | success                 |    901
 www.atsjournals.org                                             |                         |   1245
 www.atsjournals.org                                             | success                 |   1189
 www.biorxiv.org                                                 |                         |    712
 www.biorxiv.org                                                 | success                 |    670
 www.bloomsburycollections.com                                   |                         |    982
 www.bloomsburycollections.com                                   | terminal-bad-status     |    566
 www.cahiers-clsl.ch                                             |                         |    305
 www.cahiers-clsl.ch                                             | success                 |    298
 www.cairn.info                                                  |                         |   1799
 www.cairn.info                                                  | no-pdf-link             |    662
 www.cairn.info                                                  | link-loop               |    487
 www.cairn.info                                                  | success                 |    355
 www.cairn.info                                                  | terminal-bad-status     |    267
 www.cambridge.org                                               |                         |   3258
 www.cambridge.org                                               | no-pdf-link             |   1682
 www.cambridge.org                                               | success                 |    682
 www.cambridge.org                                               | bad-redirect            |    404
 www.cambridge.org                                               | link-loop               |    302
 www.dbpia.co.kr                                                 |                         |    763
 www.dbpia.co.kr                                                 | no-pdf-link             |    443
 www.dbpia.co.kr                                                 | redirect-loop           |    287
 www.degruyter.com                                               |                         |  12655
 www.degruyter.com                                               | no-pdf-link             |   9112
 www.degruyter.com                                               | success                 |   2898
 www.degruyter.com                                               | spn2-backoff            |    507
 www.dib.ie                                                      |                         |   1381
 www.dib.ie                                                      | no-pdf-link             |   1378
 www.dovepress.com                                               |                         |    231
 www.dovepress.com                                               | success                 |    216
 www.e-manuscripta.ch                                            |                         |    767
 www.e-manuscripta.ch                                            | success                 |    399
 www.e-periodica.ch                                              |                         |   1406
 www.e-periodica.ch                                              | no-pdf-link             |   1402
 www.e-rara.ch                                                   | no-pdf-link             |    251
 www.e-rara.ch                                                   |                         |    251
 www.editoracientifica.org                                       | no-pdf-link             |    205
 www.editoracientifica.org                                       |                         |    205
 www.elgaronline.com                                             |                         |    427
 www.elibrary.ru                                                 |                         |    616
 www.elibrary.ru                                                 | terminal-bad-status     |    364
 www.elibrary.ru                                                 | no-pdf-link             |    216
 www.emerald.com                                                 |                         |    862
 www.emerald.com                                                 | no-pdf-link             |    724
 www.endocrine-abstracts.org                                     |                         |   1907
 www.endocrine-abstracts.org                                     | no-pdf-link             |   1905
 www.eurekaselect.com                                            |                         |    285
 www.eurekaselect.com                                            | link-loop               |    246
 www.even3.com.br                                                |                         |    233
 www.frontiersin.org                                             |                         |    585
 www.frontiersin.org                                             | spn2-backoff            |    436
 www.humankineticslibrary.com                                    | no-pdf-link             |    207
 www.humankineticslibrary.com                                    |                         |    207
 www.igi-global.com                                              |                         |   1600
 www.igi-global.com                                              | no-pdf-link             |   1199
 www.igi-global.com                                              | bad-redirect            |    258
 www.inderscience.com                                            |                         |    385
 www.inderscience.com                                            | no-pdf-link             |    365
 www.inderscienceonline.com                                      |                         |    202
 www.ingentaconnect.com                                          |                         |    450
 www.ingentaconnect.com                                          | no-pdf-link             |    260
 www.jstage.jst.go.jp                                            |                         |   1248
 www.jstage.jst.go.jp                                            | success                 |    870
 www.karger.com                                                  |                         |    313
 www.liebertpub.com                                              |                         |    271
 www.liebertpub.com                                              | no-pdf-link             |    241
 www.nicecjournal.co.uk                                          |                         |    274
 www.nicecjournal.co.uk                                          | success                 |    274
 www.nomos-elibrary.de                                           |                         |   1771
 www.nomos-elibrary.de                                           | no-pdf-link             |    788
 www.nomos-elibrary.de                                           | redirect-loop           |    506
 www.nomos-elibrary.de                                           | bad-redirect            |    207
 www.osti.gov                                                    |                         |    381
 www.osti.gov                                                    | link-loop               |    326
 www.persee.fr                                                   |                         |    277
 www.preprints.org                                               |                         |    225
 www.preprints.org                                               | success                 |    225
 www.protocols.io                                                |                         |    770
 www.protocols.io                                                | success                 |    485
 www.repository.cam.ac.uk                                        |                         |    510
 www.repository.cam.ac.uk                                        | bad-redirect            |    213
 www.research-collection.ethz.ch                                 |                         |    416
 www.research-collection.ethz.ch                                 | bad-redirect            |    249
 www.researchsquare.com                                          |                         |   1121
 www.researchsquare.com                                          | bad-redirect            |    985
 www.scielo.br                                                   |                         |    828
 www.scielo.br                                                   | success                 |    641
 www.sciencedirect.com                                           |                         |   8567
 www.sciencedirect.com                                           | terminal-bad-status     |   5773
 www.sciencedirect.com                                           | spn2-wayback-error      |   1590
 www.sciencedirect.com                                           | no-pdf-link             |    576
 www.sciencedirect.com                                           | spn2-backoff            |    479
 www.sciendo.com                                                 |                         |    257
 www.sciendo.com                                                 | success                 |    222
 www.scitepress.org                                              |                         |    381
 www.scitepress.org                                              | no-pdf-link             |    377
 www.spiedigitallibrary.org                                      |                         |   1061
 www.spiedigitallibrary.org                                      | bad-redirect            |    571
 www.spiedigitallibrary.org                                      | gateway-timeout         |    233
 www.tandfonline.com                                             |                         |   4934
 www.tandfonline.com                                             | no-pdf-link             |   2088
 www.tandfonline.com                                             | terminal-bad-status     |   1282
 www.tandfonline.com                                             | blocked-cookie          |    757
 www.tandfonline.com                                             | redirect-loop           |    488
 www.tandfonline.com                                             | spn2-wayback-error      |    202
 www.taylorfrancis.com                                           |                         |   3979
 www.taylorfrancis.com                                           | link-loop               |   1928
 www.taylorfrancis.com                                           | no-pdf-link             |   1840
 www.techniques-ingenieur.fr                                     |                         |    354
 www.techniques-ingenieur.fr                                     | no-pdf-link             |    353
 www.thieme-connect.de                                           |                         |   1987
 www.thieme-connect.de                                           | no-pdf-link             |    949
 www.thieme-connect.de                                           | link-loop               |    869
 www.tib.eu                                                      | no-pdf-link             |    315
 www.tib.eu                                                      |                         |    315
 www.un-ilibrary.org                                             | no-pdf-link             |    352
 www.un-ilibrary.org                                             |                         |    352
 www.worldscientific.com                                         |                         |    668
 www.worldscientific.com                                         | no-pdf-link             |    629
 www.zora.uzh.ch                                                 |                         |    318
 zenodo.org                                                      |                         |  46585
 zenodo.org                                                      | no-pdf-link             |  29519
 zenodo.org                                                      | success                 |  14768
 zenodo.org                                                      | terminal-bad-status     |    810
 zenodo.org                                                      | wrong-mimetype          |    691
 zenodo.org                                                      | spn2-cdx-lookup-failure |    395
 zenodo.org                                                      | spn2-backoff            |    294
 zivahub.uct.ac.za                                               |                         |   1909
 zivahub.uct.ac.za                                               | no-pdf-link             |   1880
 zop.zb.uzh.ch                                                   |                         |    228
 zop.zb.uzh.ch                                                   | success                 |    217
                                                                 |                         | 365582
                                                                 | success                 | 141497  38.7%
                                                                 | no-pdf-link             | 120852  33.0%
                                                                 | terminal-bad-status     |  31900   8.7%
                                                                 | spn2-backoff            |  16979   4.6%
                                                                 | link-loop               |  13624   3.7%
                                                                 | bad-redirect            |   8736
                                                                 | redirect-loop           |   7405
                                                                 | gateway-timeout         |   6997
                                                                 | spn2-cdx-lookup-failure |   5146
                                                                 | spn2-wayback-error      |   3708
                                                                 | wrong-mimetype          |   2158
                                                                 | blocked-cookie          |   1942
                                                                 | spn2-error:blocked-url  |   1733
                                                                 | wayback-error           |   1063
                                                                 | spn2-error              |    647
                                                                 | spn2-error:500          |    265
                                                                 | cdx-error               |    257
(383 rows)

----

365k in 7 days is about 52k a day, which is about expected. Around 5-7% need
retries.

important changes:
- biorxiv.org: needs fix and then retries
- academic.oup.com: should probably skip
- apps.crossref.org: need to handle this in code
- arxiv.org: should retry `terminal-bad-status` on PDFs; should also add support to extract PDF link from `/abs/`
- doi.org: investigate redirect-loop and terminal-bad-status
- osf.io: not getting PDFs
- papers.ssrn.com: why are these attempted?
- publons.com: not getting PDFs; special case these?
- www.sciencedirect.com: not working at all?

smaller:
- bridges.monash.edu: fix, then retry?
- dl.acm.org: some broader retries?
- figshare.com: still some attempts, but almost all no-pdf-link
- onlinelibrary.wiley.com: getting blocked broadly?
- www.endocrine-abstracts.org: HTML content?
- www.igi-global.com: no-pdf-link