Libraryless. Click here for Pure Java version (2534L/17K/56K).
1 | !752 |
2 | |
3 | p { |
4 | S username = "stefanreich"; |
5 | Map<S, O> data = loadMediumJSON(username); |
6 | printStructureLines(data); |
7 | |
8 | pcall { |
9 | Map user = (Map) data.get("user"); |
10 | print("Name: " + user.get("name")); |
11 | print("Bio: " + user.get("bio")); |
12 | print("Image ID: " + user.get("imageid")); |
13 | // Could do more here, like Twitter name |
14 | } |
15 | |
16 | pcall { |
17 | Map references = (Map) data.get("references"); |
18 | psl("References: ", references); |
19 | Map<S, Map> posts = (Map) references.get("Post"); |
20 | psl("Posts: ", posts); |
21 | for (S postID : keys(posts)) { |
22 | print("Post " + postID); |
23 | Map post = posts.get(postID); |
24 | psl("Post: ", post); |
25 | |
26 | S uniqueSlug = getString(post, "uniqueSlug"); |
27 | S link = "https://medium.com/@" + username + "/" + uniqueSlug; |
28 | S title = getString(post, "title"); |
29 | print("Title: " + title); |
30 | print("Probable link: " + link); |
31 | //psl(post); |
32 | print(); |
33 | } |
34 | } |
35 | } |
36 | |
37 | sbool loadMediumJSON_verbose; |
38 | |
39 | static Map<S, O> loadMediumJSON(S username) { |
40 | S html = loadPageWithUserAgent("https://medium.com/@" + username, "Mac Safari"); |
41 | if (loadMediumJSON_verbose) print(html); |
42 | |
43 | S prefix = [[window["obvInit"](]]; |
44 | int i = indexOf(html, prefix); |
45 | if (i < 0) null; |
46 | int j = indexOf(html, "\n", i); |
47 | if (j < 0) null; |
48 | S json = dropSuffix(")", trim(substring(html, i+l(prefix), j))); |
49 | ret jsonDecodeMap(json); |
50 | } |
Began life as a copy of #1004101
download show line numbers debug dex old transpilations
Travelled to 14 computer(s): aoiabmzegqzx, bhatertpkbcr, cbybwowwnfue, cfunsshuasjs, ddnzoavkxhuk, gwrvuhgaqvyk, ishqpsrjomds, lpdgvwnxivlt, mqqgnosmbjvj, pyentgdyhuwx, pzhvpgtvlbxg, tslmcundralx, tvejysmllsmz, vouqrxazstgt
No comments. add comment
Snippet ID: | #1005070 |
Snippet name: | Try scraping medium.com - fixing |
Eternal ID of this version: | #1005070/1 |
Text MD5: | 2aaba4723f055b121c24afb345b9150c |
Transpilation MD5: | 39b64bf31a6b959fb74618f5c9ea3d42 |
Author: | stefan |
Category: | javax / networking |
Type: | JavaX source code |
Public (visible to everyone): | Yes |
Archived (hidden from active list): | No |
Created/modified: | 2016-10-04 21:59:11 |
Source code size: | 1431 bytes / 50 lines |
Pitched / IR pitched: | No / No |
Views / Downloads: | 580 / 642 |
Referenced in: | [show references] |