Comments (5)
Doing a few more text checks, and it looks like there are a few more fields that have leading / trailing whitespace. I figure I'll just throw other potential cleanup items on here.
unsplash_photos.exif_camera_make
unsplash_photos.exif_camera_model
unsplash_photos.photo_location_name
unsplash_conversions.keyword
unsplash_keywords.keyword
As an example:
unsplash_lite=# select '>' || keyword || '<', count(*) from unsplash_keywords where keyword IN (' wallpaper', 'wallpaper', 'wallpaper ') group by 1 order by 2 desc;
?column? | count
--------------+-------
>wallpaper< | 1951
> wallpaper< | 12
>wallpaper < | 5
from datasets.
Hey! Thank you for all this feedback!
We haven't fully normalized these location fields internally yet, that's why they're still in "free form" (we do offer Google Maps suggestions in the form but sometimes they're not being used). There are multiple challenges here (including i18n) and I don't think this is a priority for us in the near future. I would say that, in this case and for now, it'd be up to the person utilizing the dataset to do some cleaning if they want better data quality on specific fields.
The conversion_country
is a guaranteed ISO 3166-1 code (2 letter country code). The lookup is based upon the GeoLite2-City database from Maxmind.
Thank you, hopefully we can improve the dataset over time thank to everyone's feedback.
Also, of course, whenever we're implementing new things internally, it'll be reflected on the datasets.
from datasets.
@TimmyCarbone no worries, just wanting to make sure to utilize this dataset correctly. And I'll plan on doing various cleanup activities 😄
from datasets.
Let's keep this issue opened so it stays on our and everyones radar though :)
I believe that having clean fields is important. We might and will probably get to normalizing the location fields at some point in the future, it's just not really planned yet. I'll keep you and everyone posted on this issue as soon as we have a plan to tackle it.
Also, if anyone wants to give it a shot, we'd be happy to implement good solutions from the open-source community!
from datasets.
For now, we'll address the trailing/leading whitespace issue on all the fields you mentioned.
These issues will be fixed in the 1.1.0
release.
We'll keep the normalization feature aside for now since it's a much bigger project that requires more resources than we have at the moment.
from datasets.
Related Issues (20)
- Include blurhash in the dataset
- Downloading the images HOT 1
- Getting the updated dataset HOT 1
- Explanation on the ai_service_2_confidence column in keywords.tsv000 (range seems weird) HOT 2
- Download the images in Unsplash-lite
- Elaboration of data fields and additions HOT 1
- Will I be banned if I download the pictures? HOT 1
- Judie Lee Montoya HOT 1
- Wrong URL on some photos HOT 5
- Feature request: Add description = true feature to your random API requests HOT 1
- The staff Victor Ballesteros refuses to give me the access to the full dataset HOT 2
- Are there any similar datasets with records of anonymous user visits? HOT 1
- How was the LITE dataset sampled? HOT 4
- Can't download the lite data set HOT 5
- no access to the full dataset
- Apply for unsplash full data. HOT 2
- Publishing on Hugging Face
- Wrong photo_image_url + premium photos HOT 1
- Question about License
- Values of latitude and longitude entries in dataset are swapped
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datasets.