Add PubMed IDs

[gaurav]

Adds 36,980,104 PubMed IDs (PMIDs) and their titles into NodeNorm (but not into NameRes, since I don't think putting the titles in there matches our use-case). Also includes DOI and PMCID mappings. Closes #204.

To implement this, I also added a recursive download option to our FTP download option.

[cbizon]

What is the relationship between PMID and PMC? I'm assuming that the pubmed download doesn't include PMC or doi?

[gaurav]

What is the relationship between PMID and PMC? I'm assuming that the pubmed download doesn't include PMC or doi?

It does! The PubMed download consists of 1000+ baseline XML files and 100s of update XML files. Each file conforms to the PubMed DTD, and consists of a list of articles, each of which can have multiple ArticleId elements, which can provide DOIs, PMCIDs, and other identifiers. For Article IDs we assume that they are additive, i.e. if a baseline XML file has a PMID with 1 DOI and then an update XML file has the same PMID with 1 PMCID (i.e. the DOI has been removed for some reason), we would write out the PMID with both 1 DOI and 1 PMCID. But I can't think of any reason why it would be correct to remove an ArticleId in an update, so I think this is okay.

[cbizon]

So does this now allow transformation between PMID and PMCID?

[gaurav]

Yup! Also DOIs. e.g. https://nodenormalization-sri.renci.org/1.4/get_normalized_nodes?curie=PMC%3APMC8855714&curie=doi%3A10.3897%2Fzookeys.420.7089&conflate=true&drug_chemical_conflate=false&description=false

[cbizon]

Oh that's awesome! We should alert translator to this as well