Libraryless. Click here for Pure Java version (2435L/17K/54K).
!752 p { S username = "stefanreich"; Map<S, O> data = loadMediumJSON(username); printStructureLines(data); pcall { Map user = (Map) data.get("user"); print("Name: " + user.get("name")); print("Bio: " + user.get("bio")); print("Image ID: " + user.get("imageid")); // Could do more here, like Twitter name } pcall { L streamItems = (L) data.get("streamItems"); for i over streamItems: { print("Stream item #" + i); Map item = (Map) streamItems.get(i); Map preview = (Map) item.get("postPreview"); psl(preview); if (preview != null) { S postID = getString(preview, "postId"); Map post = (Map) preview.get("post"); //psl(post); print("Post ID: " + postID); S uniqueSlug = getString(post, "uniqueSlug"); S link = "https://medium.com/@" + username + "/" + uniqueSlug; S title = getString(post, "title"); print("Title: " + title); print("Probable link: " + link); psl(post); } print(); } } } sbool loadMediumJSON_verbose; static Map<S, O> loadMediumJSON(S username) { S html = loadPageWithUserAgent("https://medium.com/@" + username, "Mac Safari"); if (loadMediumJSON_verbose) print(html); S prefix = [[window["obvInit"](]]; int i = indexOf(html, prefix); if (i < 0) null; int j = indexOf(html, "\n", i); if (j < 0) null; S json = dropSuffix(")", trim(substring(html, i+l(prefix), j))); ret jsonDecodeMap(json); }
download show line numbers debug dex old transpilations
Travelled to 14 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, ddnzoavkxhuk, gwrvuhgaqvyk, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt
No comments. add comment
Snippet ID: | #1004101 |
Snippet name: | Try scraping medium.com - using streamItems, stopped working |
Eternal ID of this version: | #1004101/1 |
Text MD5: | 831f4f1a066139c88a8ef70483af2a44 |
Transpilation MD5: | 3de7994e59212667ede5c3c61ca517e3 |
Author: | stefan |
Category: | javax / networking |
Type: | JavaX source code |
Public (visible to everyone): | Yes |
Archived (hidden from active list): | No |
Created/modified: | 2016-10-04 21:48:40 |
Source code size: | 1556 bytes / 53 lines |
Pitched / IR pitched: | No / No |
Views / Downloads: | 608 / 698 |
Referenced in: | [show references] |