destinyitemmanager/dim-api

Destiny Item Manager API Service

dim-api's Issues

Delete item annotations with empty notes+tag

DIM does this automatically with tag_cleanup, but if users don't use DIM but do use DIM Sync through other sites they can leave garbage

Investigate removing Ingress Controller

Today we have a load balancer which forwards to an Nginx Ingress Controller which forwards to our application pods. It seems like we could instead have the Load Balancer directly forward to the pods? We lose the customizability of having nginx in there, but I'm not sure what else we'd lose.

Periodically delete old searches?

Perhaps we should occasionally delete old searches that haven't been saved or used in a while.

API to transfer/equip items and loadouts

Proposed change
I would like to see an API added to DIM that would allow people to access DIM from their own apps, for example switching loadouts to a saved one in DIM. This would allow people to write apps that interface with other technologies, like voice control software (e.g. Google Assistant, Alexa)

How does this fit into your workflow?
Being on console makes it annoying to have to open my inventory every time and wait for it to load, so I would love to just be able to say something like "OK Google, tell Destiny to switch to my PVP loadout". Of course it would be up to other developers to write a skill for Google Assistant or Alexa, unless you develop that too.

DIM loadouts seem to drop craftedDate

https://old.reddit.com/r/DestinyItemManager/comments/13jmccp/reshaping_weapons_marks_them_as_missing_from/

dim-api/api/db/loadouts-queries.ts

Lines 135 to 161 in d2338d0

	export function cleanItem(item: LoadoutItem): LoadoutItem {
	const hash = item.hash;
	if (!Number.isFinite(hash)) {
	throw new Error('hash must be a number');
	}
	const result: LoadoutItem = {
	hash,
	};
	if (item.amount && Number.isFinite(item.amount)) {
	result.amount = item.amount;
	}
	if (item.id) {
	if (!isValidItemId(item.id)) {
	throw new Error(`item ID ${item.id} is not in the right format`);
	}
	result.id = item.id;
	}
	if (item.socketOverrides) {
	result.socketOverrides = item.socketOverrides;
	}
	return result;
	}

Set up caching for dim.gg share pages

They need some cache headers

Bring back audit log

I should do it correctly this time, but with more sites getting onboard it'd be great to have logging of which sites did which actions.

Switch to a different docker registry (digitalocean?)

The main docker repo is closing free accounts

Impossible to use non-localhost addresses for dev server

When you get to the DIM api key part of the initial setup, if your server is remote, you get an error that says "Could not register app: InvalidRequest - Can only register apps for localhost or 127.0.0.1, your host was [your server]".

In addition to the obvious limitations, this makes it impossible to do development on a Chromebook. The linux vm that you can start is only accessible to the main chrome instance via 100.115.92.197.

(to anyone who finds this later: you also need to run yarn start with --host 0.0.0.0 --disable-host-check)

Replace private symbols with words in loadout share titles

We use special fonts for Destiny symbols, but those won't work on whatever embeds our OpenGraph preview. We should sub them out with words. Ideally localized...

Figure out how to drive complete refresh of tokens without logout

A while ago I made a change for #141 that added more info to tokens on login, so we could stop relying on bungie.net account for saved info. Since then, about half the tokens have been replaced, due to people naturally logging out and back in or refreshing tokens. However for some reason the rest haven't updated even after a month (token lifetime is one month).

Track quests and milestones

There are some quests that are account-wide that you can not track with the API because there isn't an instance ID, it would be nice to DIM-track them.

Additionally, tracking normal quest-items (steps) with instance ids by their parent quest would be nice so when you complete a quest step in-game, DIM is still tracking the questline.

Rethink index choices

I think I can probably drop a bunch of the primary key constraints in favor of fewer, more useful unique indexes.

Use Webpack or something to generate content-hashed assets

For static assets (CSS, images, etc) it'd be great to generate content-hashed & minified assets with something like Webpack. Otherwise when we cache them we'll get in trouble.

Set up Github actions to automatically publish to NPM

When files in dim-api-types changes, publish to NPM

Periodically delete unused loadout shares

40% of loadout shares are never viewed by anyone (not even the creator!). Probably just clicking the button by accident or choosing not to share. We should periodically clean these up with a postgres job.

Terminate SSL at the load balancer

Right now we use a load balancer to forward traffic to our Nginx Ingress Controller, which has a LetsEncrypt certificate for handling SSL. The LetsEncrypt certificate is managed by Kubernetes. Instead, we could have Nginx run without SSL at all, and install a LetsEncrypt cert on the load balancer directly. We could then change the LB from a TCP route to an HTTP2 (HTTPS) route. This would save SSL processing on the Ingress Controller.

Switch from Graphite to Prometheus

I'm using a Graphite/StatsD metrics collector because that's what I'm used to, but I suspect Prometheus would be better and it'd be fun to learn how to set it up and use it.

Store saved searches keyed by query hash

Right now the search table is keyed by the whole query, which really isn't worth it especially if queries are long. We should try to key off a hash of the query instead.

Loadout short URL

I'd like to build a custom "url shortener" in DIM for sharing loadouts. Rather than taking a URL as input (which would be ripe for abuse), it would take loadout parameters, and save them in the database and vend a short unique URL from them. When visited, it would redirect to the full loadout optimizer URL based on the saved params.

It'd be great to have a fun domain for this too!

Do not store data under Bungie.net membership ID

It turns out that users can relink their Destiny profiles to different Bungie.net accounts, and in fact they may do this without knowing (perhaps by choosing different login options? IDK). Unfortunately all data in DIM Sync is stored under (bungie membership id, profile membership id) keys, so this makes their data inaccessible after the change.

To fix this, we could switch to storing only by profile membership ID. Then, we'd need to change the auth token to include a list of all profile membership IDs accessible from the logged in account, and use that to validate access to each bit of data. This has one glaring downside, which is that if the user links a different profile to the same Bungie.net account, they won't see it. Remember that users can still have multiple profiles if cross-save is off! However, this may be worth it, vs "my loadouts suddenly disappeared" as a logout/login will fix it.

DIM API

DestinyItemManager/DIM#4418

This is a proposal to replace Google Drive data storage with our own custom API. Development happens here.

Process.env?

I'm confused to as why this is using process.env to load a .env file, it's purpose is not to load environment files, but to return the node environment of the process. Shouldn't you be using a package like dotenv?

Localize dim.gg share landing pages

Right now their content is English only.

Load manifest so we can show a better loadout preview for dim.gg share links

Admin Page

I'd like to set up a little admin site behind GitHub Oauth to let us take actions on API stuff. Possible actions:

1. Enable/disable auto refresh.

Set up Github actions to automatically deploy

When the service changes, automatically deploy to DigitalOcean