gv14982/async-airtable

A lightweight npm package to handle working with the Airtable API.

Async Airtable

ARCHIVED

I strongly suggest using the current official Airtable SDK as it now supports promises and I don't really have time to maintain this anymore.

AsyncAirtable is a lightweight npm package to handle working with the Airtable API.

They have an existing library, but it is callback based and can get a little klunky at times, so I wrote this one that is promise based to make your life easier 😊.

I also wrote a query builder so, instead of having to write those really annyoying filter formula strings you can just use an object like:

{
  where: {
    name: 'AsyncAirtable',
    $gte: {stars: 13}
  }
}

which will generate the following filterFormula string for you: AND({name} = 'AsyncAirtable', {stars} >= 13)

Requirements

• NodeJS
• npm
• Airtable account

Installation

• Be sure get your API key

• To get the base ID of your new base. You can do this by heading over to Airtable's API page and selecting that base from the list, you should see:

  The ID of this base is BASE_ID

• Install all dependancies:

npm install asyncairtable

Then you should be good to go!👍

Browser

If you want to use AsyncAirtable in a browser, please use the files in the ./dist folder. There is a regular and a minified version.

They are also available via unpkg.com:

• Regular
• Minified

Usage

const AsyncAirtable = require('async-airtable'); // or import { AsyncAirtable } from 'asyncairtable';
const asyncAirtable = new AsyncAirtable(API_KEY, BASE_ID, { ...CONFIG });

Documentation

To setup documentation run: npm run doc

This will generate a docs folder. Just open or serve index.html and you will have the docs!

You can also view them online.

License

MIT

async-airtable's Issues

Support for views?

Is your feature request related to a problem? Please describe.
I can't find any reference to Airtable Views in the docs, is querying a specific view within a table supported?

Describe the solution you'd like
If supported, clear documentation on how to use it. If not supported, then I would like it to be added?

Describe alternatives you've considered
I've tried a couple approaches using the standard airtable API, if it isn't supported I'll just end up writing my own wrapper for airtable, but i'd rather use a more standardized thing

🚨 Figure out a better return type for the rateLimitHandler function

Currently the asyncAirtable#rateLimitHandler method currently has a return type of Promise<any>, which is not ideal and causing a linter warning.

We need to come up with a better return type to handle this.

✨ Add TypeScript support

Setup a types file, migrate existing files to TypeScript.

Further documentation for the query builder.

Added supporting documentation for the query builder with multiple common examples (if they are supported) such as

• Querying a table and returning results based on IDs from another table. For example: I only want Events that have taken place in these Locations - Location ID: 1, Location ID 2

🎨 Change names of create, update, and delete methods.

I wasn't paying attention when writing this, and I realize using a method called "delete" is probably not a good idea.

Change the name of the create, update, and delete methods to createRecord, updateRecord, and deleteRecord.

✅ Be sure to also update all tests so they still pass.

📝 Also be sure to update the README.

Permissions Error

When calling the createRecord method with the typecast param set to true in an attempt to use the record name instead of record ID to add an entry to a linked record field in the new record I get the following error from Airtable. Is this correct behavior? From Airtable's api docs it appears that typecast alllows for them to do data conversion from string values. Thanks!

Error: Error: {"error":{"type":"INVALID_PERMISSIONS","message":"You are not permitted to perform this operation because the field or table has been configured to limit who can perform this operation"}}
at AsyncAirtable. (/rbd/pnpm-volume/167adb67-e569-4731-8084-63cd4f28a871/node_modules/.registry.npmjs.org/asyncairtable/2.1.0/node_modules/asyncairtable/lib/asyncAirtable.js:284:31)
at step (/rbd/pnpm-volume/167adb67-e569-4731-8084-63cd4f28a871/node_modules/.registry.npmjs.org/asyncairtable/2.1.0/node_modules/asyncairtable/lib/asyncAirtable.js:44:23)
at Object.next (/rbd/pnpm-volume/167adb67-e569-4731-8084-63cd4f28a871/node_modules/.registry.npmjs.org/asyncairtable/2.1.0/node_modules/asyncairtable/lib/asyncAirtable.js:25:53)
at fulfilled (/rbd/pnpm-volume/167adb67-e569-4731-8084-63cd4f28a871/node_modules/.registry.npmjs.org/asyncairtable/2.1.0/node_modules/asyncairtable/lib/asyncAirtable.js:16:58)
at processTicksAndRejections (internal/process/task_queues.js:86:5)

Using asyncairtable in a netlify function

An example of a netlify function using your library would be welcome. Not sure why but I always end up with an error saying that "SyntaxError: Cannot use import statement outside a module"
And this happens wether I use the require or import syntax...
Here is an example code:

// import { AsyncAirtable } from 'asyncairtable';
const AsyncAirtable = require('asyncairtable');

const keys = {
  atApiKey: process.env.AT_API_KEY,
  atDb: process.env.AT_DB,
}
const asyncAirtable = new AsyncAirtable(keys.atApiKey, keys.atDb);

async function getMoviesWithRelatedAsyncAirtable() {
  
  const movies = await asyncAirtable.select('Movies')
  const formattedMovies = formatData(movies);
 
  formattedMovies.forEach(async (movie) => {
    movie.DirectorFull = await getDirector(movie.Director[0]);
    if (movie && movie.Actors) {
      movie.ActorsFull = await getActors(movie.Actors);
    }
  })
  return formattedMovies;
}

async function getDirector(directorId) {
  const director = await asyncAirtable.find('Directors', directorId)
  return director.fields;
}

function getActors(actorsIds) {
  const actors = []
  actorsIds.forEach(async (actorId) => {
    const actor = await asyncAirtable.find('Actors', actorId)
    actors.push(actor.fields)
  })
  return actors;
}

function formatData(data) {
  return data.map(item => ({
    id: item.id,
    ...item.fields
  }))
}

// The actual netlify function !
exports.handler = async (event, context) => {
  try {
    const movies = await getMoviesWithRelatedAsyncAirtable();
    return {
      statusCode: 200,
      body: JSON.stringify(movies),
    };
  } catch (err) {
    console.log(err);
    return {
      statusCode: 500,
      body: JSON.stringify(err),
    };
  }
}

✨ Add upsert method

I think we should add AsyncAirtable#upsert.

This should function similar to the upsert method in many SQL ORM's:

• Check if a record exists using a filter formula.
• If it exists, update that record with the fields data passed.
• If not, create a new record with the fields data passed.

Be sure to write tests and do any error handling on it.

✨ Add browser support

Currently this package only uses node-fetch, but we should add the option to use the built in fetch API so it can be used in the browser. I'll also look at hosting it on a CDN for people to include directly in their projects.

🎨 Change the structure of the records you pass into create and update methods

Currently the structure of the data you pass into:

• AsyncAirtable#create
• AsyncAirtable#update
• AsyncAirtable#bulkCreate
• AsyncAirtable#bulkUpdate

nvm about the create functions. Just leave those as is

Is structured as:

{
  id: id // Only required for the update methods
  field1: value1,
  field2: value2
}

And this could cause an issue if the table has a column called id. So we should change the update and bulkUpdate methods to use the following:

{
  id: id,
  fields: {
    ...fields
  }
}

Be sure to update all relevant tests and documentation.

📝 Add comprehensive documentation of each method

Add comprehensive documentation on the following methods:

• select
• find
• create
• update
• delete
• bulkCreate
• bulkUpdate
• bulkDelete

The documentation should include the following:

• Method name
• Method description
• Method example
• Method arguments and types
• Method return type, and an example return value.

✨ Add option for destructive update

Airtable offers the option to do a destructive update, so if a field is not specified in the fields you send, it is automatically set to null. Implement this feature per their documentation:

To update referrals records, issue a request to the referrals endpoint. A PATCH request will only update the fields you specify, leaving the rest as they were. A PUT request will perform a destructive update and clear all unspecified cell values. The example at the right uses the non-destructive PATCH method

✨ Add support for models

Add support for data models to allow for type checking without sending a request.

ReferenceError: define is not defined

Describe the bug
After install and import, I've got this error. I've tried on multiple node version.

To Reproduce
Here a link to runkit : https://runkit.com/stevenpersia/referenceerror-define-is-not-defined

Expected behavior
Except to try your amazing tool 😃

Screenshots

Comments in typescript class definition are wrong

Describe the bug
Descriptive comments are wrong in /asyncairtable/lib/asyncAirtable.d.ts, appear to be offset.

Examples:

The description for the "find" function described the "createRecord" function.

find: (table: string, id: string) => Promise<AirtableRecord>;
    /**
     * Creates a new record on the specified table.

The description for the "createRecord" function described the "updateRecord" function.

createRecord: (table: string, record: Fields, typecast?: boolean | undefined) => Promise<AirtableRecord>;
   /**
    * Updates a record on the specified table.

etc. etc.

updateRecord: (table: string, record: AirtableUpdateRecord, opts?: updateOpts | undefined) => Promise<AirtableRecord>;
   /**
    * Deletes a record from the specified table.

Screenshots

Error: unable to locate global object

Describe the bug
Any attempt to use async-airtable in the browser via Next.js fails due to an inability to locate the global object.

To Reproduce
Steps to reproduce the behavior:

1. Visit this CodeSandbox: https://codesandbox.io/s/silly-galileo-0zt3w?file=/pages/index.js
2. Observe in pages/index.js that we are referencing the browser version of async-airtable.
3. See error message in browser preview window.

Expected behavior
I expect to access the browser version of async-airtable within a React component.

Screenshots

Desktop (please complete the following information):

• OS: macOS Big Sur
• Browser: Firefox
• Version: 84

Additional context

Next.js offers ways to retrieve data via SSR, and the node version of async-airtable works perfectly within that context. However, I'd like to make some updates to Airtable from forms within React components, and that needs to happen in the browser context. I ran into this issue while attempting to make those updates.

A bit of Googling made me think that the issue might be related to node-fetch and/or referencing this instead of globalThis. However, this stuff isn't really my wheelhouse and I wasn't able to fork and fix on my own. Thanks in advance for your time and energy!

📝 Improve docs

Just go through and cleanup the JSDoc comments and make sure there are no typos and everything is cohesive and makes sense.

SyntaxError: Cannot use import statement outside a module

Describe the bug

There's an issue when trying to use the module, it just doesn't work. Am I getting something very wrong?
I used version 2.1.0 without any problems and now I tried 2.3.1 and it just won't work.

To Reproduce
Steps to reproduce the behavior:

0. create an empty folder and cd into it
1. npm install asyncairtable
2. add import { AsyncAirtable } from 'asyncairtable'; to a blank index.js
3. node index.js

It will throw the following error:

/.../node_modules/asyncairtable/lib/asyncAirtable.js:10
import buildOpts from './buildOpts';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at Object.compileFunction (node:vm:352:18)
    at wrapSafe (node:internal/modules/cjs/loader:1031:15)
    at Module._compile (node:internal/modules/cjs/loader:1065:27)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:94:18)
    at Object.<anonymous> (.../index.js:1:23)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)

Expected behavior
The script should run without any output.

Node versions

Tested with Node 16.6.1 and 17.2.0 (using nvm)
Npm version 7.20.3

Regex FilterByFormula in Select Options not Working

Describe the bug
When I try and use a regex formula filter per Airtable's supported formula elements I get no records back. Just wanting to confirm whether or not AsyncAirtable allows for the regex formula to be included in the filterByFormula field for select options before I try and debug my code any further. Thanks!

To Reproduce
Steps to reproduce the behavior:

const selectOptions = {
fields: ['Name', 'Region'],
filterByFormula: REGEX_MATCH(ARRAYJOIN({Teachers}, ", "), "${userName}")
}

✨ Add query builder

When I use the .select method, instead of having to write a filter formula string that can get kind of weird and cumbersome, I would like to be able to just pass in an object with a where property that functions similar to SQL ORM's like Sequelize.

There is no issue. I just wanted to say that this is phenomenal work.

Sorry to clog up your Issues, but hopefully a little praise is welcome here.

First of all, Airtable is awesome. It makes playing around with little data projects really easy. But...you've taken what was easy, and you've also made it fun. Thank you so much for this awesome effort. That is all. 🙃

Implement additional Filter Formula methods into query builder.

We are currently missing a large portion of the methods within Airtable's filter formulas. Some that would take priority are:

• All of Array methods
• String methods
  • SEARCH
  • FIND

I will keep this issue open until we have 100% coverage of the filter formula spec within the query builder. I can write some tests and a tracker to handle it as well:

• Text operators
  • &
• Text functions
  • ARRAYJOIN([item1, item2, item3], separator)
  • CONCATENATE(text1, [text2, ...])
  • ENCODE_URL_COMPONENT(component_string)
  • FIND(stringToFind, whereToSearch,[startFromPosition])
  • LEFT(string, howMany)
  • LEN(string)
  • LOWER(string)
  • MID(string, whereToStart, count)
  • REPLACE(string, start_character, number_of_characters, replacement)
  • REPT(string, number)
  • RIGHT(string, howMany)
  • SEARCH(stringToFind, whereToSearch,[startFromPosition])
  • SUBSTITUTE(string, old_text, new_text, [index])
  • T(value1)
  • TRIM(string)
  • UPPER(string)
• Logical operators
  • >
  • <
  • >=
  • <=
  • =
  • !=
• Logical functions
  • AND(logical1, [logical2, ...])
  • BLANK()
  • ERROR()
  • FALSE()
  • IF(logical, value1, value2)
  • ISERROR(expr)
  • NOT(boolean)
  • OR(logical1, [logical2, ...])
  • SWITCH(expression, [pattern, result ... , default])
  • TRUE()
  • XOR(logical1, [logical2, ...])
• Numeric operators
  • -
  • -
  • *
  • /
• Numeric functions
  • ABS(value)
  • AVERAGE(number1, [number2, ...])
  • CEILING(value, [significance])
  • COUNT(number1, [number2, ....])
  • COUNTA(textOrNumber1, [number2, ....])
  • COUNTALL(textOrNumber1, [number2, ....])
  • EVEN(value)
  • EXP(power)
  • FLOOR(value, [significance])
  • INT(value)
  • LOG(number, [base])
  • MAX(number1, [number2, ...])
  • MIN(number1, [number2, ...])
  • MOD(value1, divisor)
  • ODD(value)
  • POWER(base, power)
  • ROUND(value, precision)
  • ROUNDDOWN(value, precision)
  • ROUNDUP(value, precision)
  • SQRT(value)
  • SUM(number1, [number2, ...])
  • VALUE(text)
• Date and Time Functions
  • CREATED_TIME()
  • DATEADD([date], [#], 'units')
  • DATESTR([date])
  • DATETIME_DIFF([date1], [date2], 'units')
  • DATETIME_FORMAT([date], '[specified output format]')
  • DATETIME_PARSE(date, ['input format'], ['locale'])
  • DAY([date])
  • HOUR([datetime])
  • IS_AFTER([date1], [date2])
  • IS_BEFORE([date1], [date2])
  • IS_SAME([date1], [date2], [unit])
  • LAST_MODIFIED_TIME([{field1},{field2}, ...])
  • MINUTE([datetime])
  • MONTH([date])
  • NOW()
  • SECOND([datetime])
  • SET_LOCALE([date], [locale_modifier])
  • SET_TIMEZONE([date], [tz_identifier])
  • TIMESTR([date/timestamp])
  • TONOW([date]), FROMNOW([date])
  • TODAY()
  • WEEKDAY(date, [startDayOfWeek])
  • WEEKNUM(date, [startDayOfWeek])
  • WORKDAY(startDate, numDays, [holidays])
  • WORKDAY_DIFF(startDate, endDate, [holidays])
  • YEAR([date])
• Array functions
  • ARRAYCOMPACT(values)
  • ARRAYFLATTEN(values)
  • ARRAYJOIN(values, separator)
  • ARRAYUNIQUE(values)
• Record functions
  • CREATED_TIME()
  • RECORD_ID()
• REGEX functions
  • REGEX_MATCH(string, regex)
  • REGEX_EXTRACT(string, regex)
  • REGEX_REPLACE(string, regex, replacement)

Reference #52 as the prioritized methods are required for what they are asking for.

✨ Add logic for rate limiting

When I make a request using asyncAirtable and it receives a rate limited response, retry the the request after waiting 30 seconds.

From the Airtable documentation:

The API is limited to 5 requests per second per base. If you exceed this rate, you will receive a 429 status code and will need to wait 30 seconds before subsequent requests will succeed.

"+" character not supported in where query

Describe the bug
Query not working on emails that include "+" character.

To Reproduce
Steps to reproduce the behavior:

1. Create an airtable text column
2. Add a string like "[email protected]"
3. Attempt to query for the string:

  const records = await Airtable.select('Table_Name, {
    where: {
      primary_email: "[email protected]",
    }
  });

4. See that no records are returned.
5. Remove the "+" from the email in the query and in Airtable, try again, records are returned.

Expected behavior
1 record returned for the row w/ column containing the string.

Additional context

Also tried using encodeURIComponent and filterByFormula, neither worked.

  const records = await Airtable.select('Table_Name, {
    where: {
      primary_email: encodeURIComponent("[email protected]"),
    }
  });

✨ Add support for a filter formula or query on the update methods.

We should add support for a filter formula string or query on the update methods, so it doesn't have to be done by ID.

Something like:

asyncAirtable.update('table', {
  where: "{name} = 'Graham'" // This is just a standard filter formula string, and will need to be updated when we build the query builder too, so we don't have a fragmented user experience
  fields: {
    hungry: false
  }
}); 

Then be sure to update all tests and documentation.

Query Object Capabilities

Is something like this possible in order to get a more complex query formula for the select options when using the select method?

{
  "where": {
    "$and": [
      {
        "$or": [
          "{$eq: { '': ''}",
          "{$eq: { '': ''}",
          "{$eq: { '': ''}"
        ]
      }
    ]
  },
  "sort": [
    {
      "field": "",
      "direction": "asc"
    },
    {
      "field": "",
      "direction": "desc"
    }
  ]
}

This errors for me saying the filter formula is invalid. Also, should the error messages be saying exactly what's wrong with the formula, or are they all generic in this way? Thanks!

Uncaught Iteration Timeout errors from Airtable

I've been seeing quite a few instances of the following error in the monitoring software we use for vacfind:

Message: Error: Error: {"error":{"type":"LIST_RECORDS_ITERATOR_NOT_AVAILABLE"}}
at this.select/< line 1, column 6350 (https://vacfind.org/assets/js/asyncAirtable-beta.min.js:1)
at s line 1, column 4770 (https://vacfind.org/assets/js/asyncAirtable-beta.min.js:1)

I'm working on getting the included sourcemaps to process correctly, but this seems to me to be a relatively straightforward case of an UnhandledPromiseRejection.

Searching online leads me to this issue that mentions this error message and refers to the Airtable API docs which state that:

Iteration may timeout due to client inactivity or server restarts. In that case, the client will receive a 422 response with error message LIST_RECORDS_ITERATOR_NOT_AVAILABLE. It may then restart iteration from the beginning.

I have not personally found a way to reproduce this issue since the docs seem to say that it only happens in cases of timeouts or the airtable server going down. Although airtable seems to briefly (<30 sec) go down somewhat frequently, this isn't something that I am able to reproduce intentionally.

Would you accept a PR to add in a handler to catch these 422 LIST_RECORDS_ITERATOR_NOT_AVAILABLE errors, and If so, how would you prefer to have AsyncAirtable handle it?