Giter Site home page Giter Site logo

crossref-rs's Introduction

Crossref-rs - A rust client for the Crossref-API

Build Status Crates.io Documentation

Crossref API docs

This client is inspired by sckott/habanero.

Crossref - Crossref search API. The Crossref crate provides methods matching Crossref API routes:

  • works - /works route
  • members - /members route
  • prefixes - /prefixes route
  • funders - /funders route
  • journals - /journals route
  • types - /types route
  • agency - /works/{doi}/agency get DOI minting agency

Usage

Create a Crossref client:

let client = Crossref::builder().build()?;

If you have an Authorization token for Crossref's Plus service:

let client = Crossref::builder()
    .token("token")
    .build()?;

Encouraged to use the The Polite Pool:

Good manners = more reliable service

To get into Crossref's polite pool include a email address

let client = Crossref::builder()
     .polite("[email protected]")
     .token("your token")
     .build()?;

Constructing Queries

Not all components support queries and there are custom available parameters for each route that supports querying. For each resource components that supports querying there exist a Query struct: WorksQuery, MembersQuery, FundersQuery. The WorksQuery also differs from the others by supporting deep paging with cursors and field queries.

otherwise creating queries works the same for all resource components:

let query = WorksQuery::new("Machine Learning")
    // field queries supported for `Works`
    .field_query(FieldQuery::author("Some Author"))
    // filters are specific for each resource component
    .filter(WorksFilter::HasOrcid)
    .order(Order::Asc)
    .sort(Sort::Score);

Get Records

See this table for a detailed overview of the major components.

There are 3 different targets:

  • standalone resource components: /works, /members, funders, prefixes, types that return a list list of the corresponding items and can be specified with queries
  • Resource component with identifiers: /works/{doi}?<query>,/members/{member_id}?<query>, etc. that returns a single item if found.
  • combined with the works route: The works component can be appended to other resources: /members/{member_id}/works?<query> etc. that returns a list of matching Work items as WorkList.

This resembles in the enums of the resource components, eg. for Members:

pub enum Members {
    /// target a specific member at `/members/{id}`
    Identifier(String),
    /// target all members that match the query at `/members?query...`
    Query(MembersQuery),
    /// target a `Work` for a specific member at `/members/{id}/works?query..`
    Works(WorksIdentQuery),
}

Examples

All options are supported by the client:

Query Single Item by DOI or ID

Analogous methods exist for all resource components

let work = client.work("10.1037/0003-066X.59.1.29")?;

let agency = client.work_agency("10.1037/0003-066X.59.1.29")?;

let funder = client.funder("funder_id")?;

let member = client.member("member_id")?;

Query

let query = WorksQuery::new("Machine Learning");

// one page of the matching results
let works = client.works(query)?;

Alternatively insert a free form query term directly

let works = client.works("Machine Learning")?;

Combining Routes with the Works route

For each resource component other than Works there exist methods to append a WorksQuery with the ID option /members/{member_id}/works?<query>?

use crossref::*;
fn run() -> Result<()> {
    let client = Crossref::builder().build()?;
    let works = client.member_works(WorksQuery::new("machine learning")
    .sort(Sort::Score).into_ident("member_id"))?;
    Ok(())
}

This would be the same as using the [Crossref::works] method by supplying the combined type

use crossref::*;
fn run() -> Result<()> {
    let client = Crossref::builder().build()?;
    let works = client.works(WorksQuery::new("machine learning")
     .sort(Sort::Score)
     .into_combined_query::<Members>("member_id"))?;
    Ok(())
}

** Deep paging for Works ** Deep paging results Deep paging is supported for all queries, that return a list of Work, WorkList. This function returns a new iterator over pages of Work, which is returned as bulk of items as a WorkList by crossref. Usually a single page WorkList contains 20 items.

Example

Iterate over all Works linked to search term Machine Learning

use crossref::{Crossref, WorksQuery, Work};
fn run() -> Result<(), crossref::Error> {
    let client = Crossref::builder().build()?;
    
    let all_works: Vec<Work> = client.deep_page(WorksQuery::new("Machine Learning")).flat_map(|x|x.items).collect();
    
    Ok(())
}

Which can be simplified to

use crossref::{Crossref, WorksQuery, Work};
fn run() -> Result<(), crossref::Error> {
    let client = Crossref::builder().build()?;
    
    let all_works: Vec<Work> = client.deep_page("Machine Learning").into_work_iter().collect();
    
    Ok(())
}

Iterate over all the pages (WorkList) of the funder with id funder id by using a combined query. A single WorkList usually holds 20 Work items.

use crossref::{Crossref, Funders, WorksQuery, Work, WorkList};
fn run() -> Result<(), crossref::Error> {
    let client = Crossref::builder().build()?;
    
    let all_funder_work_list: Vec<WorkList> = client.deep_page(WorksQuery::default()
            .into_combined_query::<Funders>("funder id")
      )
        .collect();
    
    Ok(())
}

Iterate over all Work items of a specfic funder directly.

use crossref::{Crossref, Funders, WorksQuery, Work, WorkList};
fn run() -> Result<(), crossref::Error> {
    let client = Crossref::builder().build()?;
    
    let all_works: Vec<Work> = client.deep_page(WorksQuery::default()
         .into_combined_query::<Funders>("funder id"))
         .into_work_iter()
         .collect();
    
    Ok(())
}

Command Line Application

Installation

cargo install crossref --features cli

Usage

Top level subcommands

USAGE:
    crossref <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    funders     Query crossref funders
    help        Prints this message or the help of the given subcommand(s)
    journals    Query crossref journals
    members     Query crossref members
    prefixes    Query crossref prefixes
    types       Query crossref types
    works       Query crossref works

Additional options for the component subcommands (querying, sorting, ordering and limiting is only supported for subcommands <works|funders|members> and is overridden by a present --id options)

USAGE:
    crossref works [FLAGS] [OPTIONS] [SUBCOMMAND]

FLAGS:
    -a, --append       if the output file already exists, append instead of overwriting the file
    -d, --deep-page    Enable deep paging. If a limit is set, then the limit takes priority.
    -h, --help         Prints help information
    -s, --silent       do not print anything
    -V, --version      Prints version information

OPTIONS:
    -i, --id <id>                    The id of component.
    -l, --limit <limit>              limit the amount of results
        --offset <offset>            Sets an offset where crossref begins to retrieve items.
        --order <order>              How to order the results: asc or desc
    -o <output>                      output path where the results shall be stored
        --polite <polite>            The email to use to get into crossref's polite pool
    -q, --query <query_terms>...     The free form terms for the query
        --sample <sample>            Request randoms Elements. Overrides all other options.
        --sort <sort>                How to sort the results, such as updated, indexed, published, issued
        --token <token>              The token to use for the crossref client
        --user-agent <user_agent>    The user agent to use for the crossref client

Examples

Retrieve a specific work by a doi

 crossref works --id "10.1037/0003-066X.59.1.29"

Save the results as json

 crossref works --id "10.1037/0003-066X.59.1.29" -o output.json

Retrieve any other components by their ids

 crossref <works|journals|members|prefixes|types> --id "10.1037/0003-066X.59.1.29" -o output.json

Some components support additional filtering

crossref <works|funders|members> --query "A search term such as `Machine learning` for works" --limit 10 --offset 200 --order asc

Get Works of a specific component, such as a member with the id 98:

crossref works member 98

<prefix|funder|prefix|type are also supported in the same way.

By default deep paging is disabled, hence the max amount of results of Works will be 20 (a single crossref page). By enabling the --deep-page flag, all available results will be gathered.

To get in to the polite pool supply your email to the request headers with --polite "[email protected]"

License

Licensed under either of these:

crossref-rs's People

Contributors

mattsse avatar she3o avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

crossref-rs's Issues

Make the underlying reqwest client shareable

Currently the reqwest client is owned directly by the crossref client, which makes it not shareable across multiple instances or in any other way.

Fix would be to simply wrap it inside a std::rc::Rc.

Cannot request works

Hey, I tried using the API.
This minimal example:

use crossref::*;

fn main() -> Result<()> {
    let client = Crossref::builder().build()?;
    let works =
        client.works(WorksQuery::new("Itech").field_query(FieldQuery::author("Felix Knorr")))?;
    Ok(())
}

Will produce

felix@ryzen-tr:~/tmp/crossrefexample$ cargo run
   Compiling crossrefexample v0.1.0 (/home/felix/tmp/crossrefexample)
warning: unused variable: `works`
 --> src/main.rs:5:9
  |
5 |     let works =
  |         ^^^^^ help: if this is intentional, prefix it with an underscore: `_works`
  |
  = note: `#[warn(unused_variables)]` on by default

warning: 1 warning emitted

    Finished dev [unoptimized + debuginfo] target(s) in 2.20s
     Running `target/debug/crossrefexample`
Error: Error { ctx: 

expected response item of type work-list but got validation-failure }

This will also happen if I add an email address via the builder

New Crossref REST API coming soon

Hi,

Thank you for maintaining one of the documented libraries for using the Crossref REST API.

We’ve been working on a new version of the REST API, replacing the Solr backend with Elasticsearch and moving from our own hardware in a datacenter to a cloud platform.

We plan to cutover to the new version shortly (expect an official announcement on our blog in the next few days with more details), and wanted to invite you to test it out before the official cutover.

Please check it out at https://api.production.crossref.org/

During the cutover phase (expected to last a few weeks), traffic will be redirected to the above domain on a pool by pool basis. Once all traffic is using the new service, we will continue to use the api.crossref.org domain, so please do not update anything to use the temporary domain.

Let me know if you have any questions. Issues can be filed into our GitLab issue repository, or I’ll keep an eye on this thread.

Thanks again,
Patrick

panic when getting work

So great to find this crate! Rust grows on me daily :-)

I am trying some very simple test code, using your example paper:

    let client = Crossref::builder().build().unwrap();
    let work = client.work("10.1037/0003-066X.59.1.29").unwrap();
    dbg!(work);

But I get a panic (backtrace):

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { ctx: stack backtrace:
   0: failure::backtrace::internal::InternalBacktrace::new::he135e868fd74a8c9 (0x10e909505)
             at /Users/mm6/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/internal.rs:44
   1: failure::backtrace::Backtrace::new::h631e316463fa96ab (0x10e90a1ce)
             at /Users/mm6/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/mod.rs:111
   2: <failure::context::Context<D>>::new::h267f1c55171969ba (0x10e340991)
             at /Users/mm6/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/context.rs:84
   3: <crossref::error::Error as core::convert::From<crossref::error::ErrorKind>>::from::hb7f3ce0c6afe4e5a (0x10e32bfbc)
             at /Users/mm6/.cargo/registry/src/github.com-1ecc6299db9ec823/crossref-0.1.0/src/error.rs:77
   4: <T as core::convert::Into<U>>::into::h07cc81735444d9cc (0x10e32a60c)
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/convert.rs:441
   5: <crossref::error::Error as core::convert::From<serde_json::error::Error>>::from::h5ce4f05b21aa4fe9 (0x10e32c09c)
             at /Users/mm6/.cargo/registry/src/github.com-1ecc6299db9ec823/crossref-0.1.0/src/error.rs:89
   6: crossref::Crossref::get_response::hb6eef7e1b26e8a72 (0x10e336516)
             at /Users/mm6/.cargo/registry/src/github.com-1ecc6299db9ec823/crossref-0.1.0/src/lib.rs:288
   7: crossref::Crossref::work::h45e70744a81dc196 (0x10e3369a2)
             at /Users/mm6/.cargo/registry/src/github.com-1ecc6299db9ec823/crossref-0.1.0/src/lib.rs:356
   8: main::main::hb5c6dd26f5ebe32c (0x10e282442)
             at src/bin/main.rs:15
   9: std::rt::lang_start::{{closure}}::hc2db45f29c6f06e4 (0x10e27f9a1)
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  10: std::panicking::try::do_call::hbdc2ad4ee791d633 (0x10e95ff27)
             at src/libstd/panicking.rs:297
  11: ___rust_maybe_catch_panic (0x10e96373e)
             at src/libpanic_unwind/lib.rs:92
  12: std::rt::lang_start_internal::h3dc68cf5532522d7 (0x10e960a1c)
             at src/libstd/rt.rs:48
  13: std::rt::lang_start::hcad8a588769f53fd (0x10e27f981)
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  14: _main (0x10e282721)

invalid serde: invalid type: floating point `1`, expected i32 }', src/libcore/result.rs:997:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:70
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:200
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:215
   4: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:478
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:385
   6: std::panicking::try::do_call
             at src/libstd/panicking.rs:312
   7: core::char::methods::<impl char>::escape_debug
             at src/libcore/panicking.rs:85
   8: core::result::unwrap_failed
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/macros.rs:16
   9: <core::result::Result<T, E>>::unwrap
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/result.rs:798
  10: main::main
             at src/bin/main.rs:15
  11: std::rt::lang_start::{{closure}}
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  12: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:297
  13: panic_unwind::dwarf::eh::read_encoded_pointer
             at src/libpanic_unwind/lib.rs:92
  14: std::panicking::update_count_then_panic
             at src/libstd/panicking.rs:276
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  15: std::rt::lang_start
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  16: main::main

Add deep paging support

Deep paging allows for retrieving large amounts of work entries via several requests each pointing to a specific index in the matching data set on crossref.

To achieve deep paging, a special token has to be set and passed as parameter.
Support should be like an Interator that returns the work entries of a response each iteration

work query returns serde error

I am trying
https://docs.rs/crossref/0.1.0/crossref/struct.Crossref.html#method.work
with DOI "10.1371/journal.ppat.1007512" (code from master branch)

Searching on the CrossRef site gets the correct work, but from the work method I get:

Err(Error { ctx:
invalid serde: invalid length 0, expected struct Relation with 3 elements })

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.