Giter Site home page Giter Site logo

axe312ger / sqip Goto Github PK

View Code? Open in Web Editor NEW
3.4K 23.0 82.0 26.98 MB

"SQIP" (pronounced \skwɪb\ like the non-magical folk of magical descent) is a SVG-based LQIP technique.

Home Page: http://axe312ger.github.io/sqip

License: The Unlicense

JavaScript 5.73% TypeScript 94.27%
webperf webperformance performance image images placeholder lqip svg user-experience perceived-performance

sqip's Introduction

This is the v1 alpha readme. You can find the current docs here.

SQIP - a pluggable image converter with vector support

npm npm npm

CircleCI codecov Maintainability Contributor Covenant

SQIP is a flexible, and a little bit different image processor. It is available as node API and CLI.

By combining plugins you can use it for several purposes:

  • Create super-tiny image previews to improve your websites lazy loading experience
  • Do art by converting images into abstract representations of themselfes
  • Quickly convert, resize or optimize a set of pixel or vector images
  • More? Ideas, contributions and community plugins are very welcome

Table of contents

Examples

Get a more detailed look on our demo website.

Requirements

Non-64bit operating systems requirements

The most common plugin sqip-plugin-primitive is packed with a 64bit executable for all 3 major operating systems. Users with non 32-bit operating system or those who simply want to use the latest and greatest verison of primitive need:

After installing Primitive, you may also need to add the path to the Primitive binary file.

For macOS

It would generally look something like

/Users/myMacbook/go/bin

To do this on a Mac, type: sudo vim /etc/paths into your terminal, and add the path to your Primitive binary file, but be sure to add the full path, /Users/<username>/go/bin and not ~/go/bin.

For PC

Using the command line (https://www.windows-commandline.com/set-path-command-line) Using a GUI (https://www.computerhope.com/issues/ch000549.htm)

Node

CLI see here

Installation

You need the core plugin sqip plus all the plugins you want to use like sqip-plugin-primtive, sqip-plugin-svgo and more.

For example:

npm install sqip@canary sqip-plugin-primitive@canary sqip-plugin-svgo@canary sqip-plugin-data-uri@canary

This is the v1 alpha readme. Click here for v0 "stable" instructions.

Hint: SQIP is plugin based, you might want to install more plugins later on. See Plugins section.

Usage

SQIP is async.

try {
  const result = await sqip({...options})
  console.log(result)
} catch (err) {
  console.error(err)
}

// or

sqip({...options})
  .then(result => console.log(result))
  .catch(error => console.error(error))

If you passed a single image to process, SQIP will return the following result object:

{
  content: Buffer.from('<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 300 188">...</svg>'),
  metadata: {
    originalWidth: 1024,
    originalHeight: 640,
    palette: {
      Vibrant: Vibrant.Swatch,
      DarkVibrant: Vibrant.Swatch,
      LightVibrant: Vibrant.Swatch,
      Muted: Vibrant.Swatch,
      DarkMuted: Vibrant.Swatch,
      LightMuted: Vibrant.Swatch
    },
    width: 300,
    height: 188,
    type: 'svg',
    // These will be added by sqip-plugin-data-uri
    dataURI: "data:image/svg+xml,...",
    dataURIBase64: 'data:image/svg+xml;base64,...'
  }
}

Documentation for all 6 colors from the palette: Vibrant.Swatch

Plugins might add their own meta data

Multiple input images will result in an array of result objects.

Process folder with default settings

import { sqip } from 'sqip'
import { resolve } from 'path'

;(async () => {
  try {
    // Process whole folder with default settings
    const folderResults = await sqip({
      input: resolve(__dirname, 'images/originals'),
      output: resolve(__dirname, 'images/previews')
    })
    console.log(folderResults)
  } catch (err) {
    console.log('Something went wrong generating the SQIP previews')
    console.error(err)
  }
})()

Use custom plugin config

This will run:

  • Primitive with custom settings
  • SVGO with default settings
;(async () => {
  const pluginResults = await sqip({
    input: resolve(__dirname, 'images/originals'),
    output: resolve(__dirname, 'images/previews'),
    plugins: [
      {
        name: 'sqip-plugin-primitive',
        options: {
          numberOfPrimitives: 8,
          mode: 0,
        },
      },
      'sqip-plugin-svgo',
    ],
  })
  console.log(pluginResults)
})()

For further configuration options see here

CLI

Installation

npm install -g sqip-cli@canary

This is the v1 alpha readme. You can find the current docs here.

Usage examples

Using the help efficently

Make sure to specify plugins when using --help to see the available plugin options.

sqip -h -p primitive -p blur -p svgo
Result:
sqip CLI

  Usage: sqip --input [path]

  "SQIP" (pronounced skwɪb like the non-magical folk of magical descent) is a
  SVG-based LQIP technique - https://github.com/technopagan/sqip

Options

  -h, --help                                  Show help
  --version                                   Show version number
  -p, --plugins string[]                      One or more plugins. E.g. "-p primitive blur"
  -i, --input string
  -o, --output string                         Save the resulting SVG to a file. The svg result will be returned by default.
  -n, --primitive-numberOfPrimitives number   The number of primitive shapes to use to build the SQIP SVG
  -m, --primitive-mode number                 The style of primitives to use:
                                              0=combo, 1=triangle, 2=rect, 3=ellipse, 4=circle, 5=rotatedrect, 6=beziers,
                                              7=rotatedellipse, 8=polygon
  -b, --blur-blur number                      Set the GaussianBlur SVG filter value. Disable it via 0.

Examples

  Output input.jpg image as SQIP
  $ sqip --input /path/to/input.jpg

  Save input.jpg as result.svg with 25 shapes and no blur
  $ sqip -i input.jpg -n 25 -b 0 -o result.svg

Process single file

$ sqip -i __tests__/fixtures/beach.jpg
Processing: __tests__/fixtures/beach.jpg
[Preview image (iTerm2 users only)]
┌───────────────┬────────────────┬───────┬────────┬──────┐
│ originalWidth │ originalHeight │ width │ height │ type │
├───────────────┼────────────────┼───────┼────────┼──────┤
│ 1024          │ 640            │ 300   │ 188    │ svg  │
└───────────────┴────────────────┴───────┴────────┴──────┘
┌─────────┬─────────────┬──────────────┬─────────┬───────────┬────────────┐
│ Vibrant │ DarkVibrant │ LightVibrant │ Muted   │ DarkMuted │ LightMuted │
├─────────┼─────────────┼──────────────┼─────────┼───────────┼────────────┤
│ #dd852f │ #be4e0c     │ #f2b17a      │ #5c8fa4 │ #694e35   │ #cfc8b7    │
└─────────┴─────────────┴──────────────┴─────────┴───────────┴────────────┘
Process multiple files via glob and use custom plugin config
sqip -p primitive -p blur -p svgo \
-i "demo/*.jpg" \
-b 6

For further configuration options see here

Config

The configuration consists of three parts. A required input, an optional output path and a configuration of plugins to be applied on the images.

input - required

Input file or directory. Supports feature rich globbing via micromatch.

CLI usage: -i/--input

output

If set, the output will be written to the given file or directory.

Otherwise, results will be output to CLI

CLI usage: -o/--output

width

Set the width of the resulting image. Negative values and 0 will fall back to the original image width.

CLI usage: -w/--width

plugins

Default: ['primitive', 'svgo']

Array of plugins. Either as a string (default config will be applied) or as a config object.

Example:

await sqip({
  ...
  plugins: [
    {
      name: 'sqip-plugin-primitive',
      options: {
        numberOfPrimitives: 8,
        mode: 0,
      },
    },
    `sqip-plugin-svgo`,
  ],
})

CLI usage:

-p/--plugins

  • Can be specified multiple times: -p svgo -p blur
  • If prefix was skipped, plugin names will be transformed to: sqip-plugin-[name]
  • To set plugin options, see plugin specifc config

Plugin specific config

  • See the Plugins section for a list of available plugins.
  • List all plugins subcommands by adding the plugin plus using the help parameter. For example: -p blur -p svgo -h will list you all options of the blur and the svgo plugins.
  • Follows the pattern --[plugin-name]-[option]=[value]

Example:

Set blur option of blur plugin to 3. You could use the -b shortcut as well.

sqip -i foo.jpg -p primitive -p blur -blur-blur 3

--parseable-output

non-TTY consoles and when the --parseable-output input flag is set, the output will be the following:

$ sqip -i __tests__/fixtures/beach.jpg --parseable-output
Processing: __tests__/fixtures/beach.jpg
originalWidth originalHeight width height type
1024          640            300   188    svg
Vibrant DarkVibrant LightVibrant Muted   DarkMuted LightMuted
#dd852f #be4e0c     #f2b17a      #5c8fa4 #694e35   #cfc8b7

--silent

No output at all on STDOUT. The process will still return an error code & message when something failed.

--print

Outputs resulting SVG to STDOUT. Ignores --silent and works with --parseable-output.

Plugins

SQIP comes with some core plugins, the community is very welcome to contribute their own plugins to SQIP. The effort to implement a tool or script doing something with images into SQIP is very minimal.

Core plugins

Here is a list of all current core plugins:

Debugging

If something is not going as expected, adding debug output might help a lot. You can achieve this by setting the DEBUG environment variable to sqip*.

On a *NIX environment, you might do the following:

DEBUG=sqip* node myscript.js

# or for CLI:

DEBUG=sqip* sqip --input...

Background & reseach about image placeholder & previews

Image placeholders are a thing: from grey boxes in skeleton screens over boxes that show the predominant color of the image that will later occupy the space and CSS color gradients made from two dominant colors up to an actual low quality raster images downscaled to a few pixels, saved in low quality and then blurred to provide a preview of image contents.

Many major players have adopted one of these image placeholder techniques: Guypo incepted LQIP in 2012 and Akamai adopted it as part of their image optimization tools, Google started using colored placeholders a long time ago, Facebook, Pinterest and Medium made a significant impact on their LQIP implementations and the most popular JS libraries for responsive images include LQIP implementations.

Overview of Image Placeholder Techniques Overview of Image Placeholders

On the low end of the bytesize spectrum of image placeholder implementations, we have skeleton screens and colored boxes, weighing only a few extra bytes each, but providing no preview of image contents. On the high end of the bytesize spectrum, the LQIP technique ships an actual raster image, which gives a good initial impression of image contents to come, but weighs more heavily in bytesize.

If we disregard Facebooks's native-app implementation of shipping a custom image decoder that enables them to hardcode image headers, the current minimum bytesize for LQIP raster images is ~400-600 bytes. At this byterange, the preview image often looks distorted and coarse, especially on HiDPI screens. Many other LQIP implementations go for preview images of ~2kb in size, which provides a much better initial visual impression but comes at the cost of significantly increased bytesize for the LQIP implementation.

SQIP is an attempt to find a balance between these two extremes: it makes use of Primitive to generate a SVG consisting of several simple shapes that approximate the main features visible inside the image, optimizes the SVG using SVGO and adds a Gaussian Blur filter to it. This produces a SVG placeholder which weighs in at only ~800-1000 bytes, looks smooth on all screens and provides an visual cue of image contents to come.

Contributing

Contributor Covenant

Before contribution, please make sure to read the contribution guidelines guidelines and the code of conduct.

Pull requests, forks and stars are always welcome. For bugs and feature requests, please create an issue.

Credits

License

This is free and unencumbered software released into the public domain.

Anyone is free to copy, modify, publish, use, compile, sell, or distribute this software, either in source code form or as a compiled binary, for any purpose, commercial or non-commercial, and by any means.

In jurisdictions that recognize copyright laws, the author or authors of this software dedicate any and all copyright interest in the software to the public domain. We make this dedication for the benefit of the public at large and to the detriment of our heirs and successors. We intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this software under copyright law.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

For more information, please refer to http://unlicense.org/

sqip's People

Contributors

aahill50 avatar area73 avatar axe312ger avatar bezoerb avatar dougglez avatar efegurkan avatar gabrielperales avatar hafizalfaza avatar jowy avatar koszta avatar maxnordlund avatar mntnorv avatar preda-bogdan avatar renovate-bot avatar renovate[bot] avatar stefanjudis avatar technopagan avatar vvo avatar vxna avatar werehamster avatar xerc avatar zcei avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sqip's Issues

Node library without CLI

In order to ease imlementation in other tools it would be helpful to expose a node library separate from the command line tool.

How to use SQIP effectively for multiple files ?

Hi,

I'm looking for the most effective way to use SQIP for all images in a directory.

I've managed to make it work running a Windows CMD:
for %x in (*) do sqip -o ./svg/%x.svg %x

But when running it in a .bat file, it doesn't work:
for %%x in (*) do sqip -o ./svg/%%x.svg %%x

Ideally, I'd like to use the Node API in my Vue/Nuxt website but I don't understand how to do that.
I'm new to dev and would need some help to make the best use of SQIP.

Thanks in advance.

Tune Primitive to create closer similarity with fewer shapes

Currently, Primitive is basically an art project: it uses a random seed as part of its SVG shape generation and is not conscious about the number of shapes it uses to achieve a good similarity.

Once we've ported Primitive (#6), we should consider changing its inner workings to be more efficient in achieving a better similiarity to the input image with fewer shapes. Additionally, we shoud consider removing the random seed so that Primitive's output becomes more deterministic for each input image.

Win64 error in sqip-plugin-primitive (requiring golang)

Hello, when i run the Skip with the following code:

sqip({
    input: 'in/',
    output: 'out/',
})
.then(result => console.log(result))
.catch(error => console.error(error));

The catch receives this message:

Please ensure that Primitive (https://github.com/fogleman/primitive, writ ten in Golang) is installed and globally available

Looking the code, the error is caused bacause the file extension .exe is not present in the file path at primitivePath variable

Not blurring all images.

When running a batch of images, a small selection of SVGs will not actually get blurred. The SVG's that don't get blurred may actually get blurred on the next batch and visa versa.

Screenshot

SVG Content:

data:image/svg xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 750 937'><path fill='#5e5145' d='M0 0h750v937H0z'/><path fill='#ffceae' fill-opacity='.5' d='M455.7 108l7.3 600.2L31.1 561.8z'/><ellipse fill-opacity='.5' rx='1' ry='1' transform='matrix(95.1777 101.02218 -375.52495 353.79954 302.3 76.1)'/><path fill='#fff1cf' fill-opacity='.5' d='M441 979l113.5-292.7L357 752.2z'/><ellipse fill='#00110b' fill-opacity='.5' rx='1' ry='1' transform='matrix(-118.08246 -24.86798 90.92995 -431.76942 634.4 845.2)'/><ellipse fill-opacity='.5' rx='1' ry='1' transform='matrix(-56.65586 45.16032 -103.38974 -129.70753 285.9 891.6)'/><path fill='#370000' fill-opacity='.5' d='M287.3 518h73.2v245h-73.2z'/><path fill='#fffff2' fill-opacity='.5' d='M130 543.5l164.6 172L258 499.7z'/><path fill='#baf8ff' fill-opacity='.5' d='M629 289.5l-2.5-73.2 124.3-4.3 2.6 73.1z'/><ellipse fill='#000002' fill-opacity='.5' rx='1' ry='1' transform='matrix(-125.36617 -5.50313 3.40138 -77.48636 645.6 36)'/><path fill='#001109' fill-opacity='.5' d='M382.6 434.2l33.7-60.8 134.5 74.5-33.8 60.8z'/></svg>

Option to choose only polygons/squares/ovals etc

Just a thought that it could be just awesome to generate art-style previews, that Primitive is capable of.
Right now, as I understand, SQIP chooses appropriate mix of different shapes?
May be it's possible to restrict generation to only triangles etc?

How to use sqip with PNG's with transparent backgrounds?

Hello folks and thanks for this great project!

While it works excellent in most of the cases, there is the situation of PNG's with transparent backgrounds, which are turned into ugly looking LQIP's that have a large dark/black area around them.

After digging several hours into this, with no real results, I decided to open this issue.

Here's an example so you can better understand the issue:

Original image (taken from fogleman/primitive#54 and also avilable here):

beers

I have sqip installed on a Debian machine and after running

sqip -n 100 -m 0 -bg #ffffff00 source.png -o target.svg I get this result (I can't upload SVG's unfortunately, so I took a snapshot):

image

https://dev.shortpixel.ai/spai/q_lqip/https://another.dobrestii.ro/wp-content/uploads/2021/01/beers.png?7

I found the #ffffff00 format in here fogleman/primitive#54, but even if the result seems to have a white background when it is rendered, it is still annoyingly black on the exterior. And it's really far away from the original image!

The desired output would be a lighter color (I do not especially expect transparency, even if that would be really nice), but the black that you see, looks really bad.

Do you have any ideas on how to make this happen? Should I use other parameters? Any suggestions would be highly appreciated! Thank you!

Create a migration guide

The API changed for node and for the CLI

We should provide an migration guide, let's gather breaking changes here for now.

  • file became input
  • node is now async
  • ...?

Error in parsing SVG: Unbound namespace prefix: "xlink"

Hello,

I try to convert PNG pictures to SVG using the following command:

node_modules/.bin/sqip -p primitive svgo -n 10 -m 2 --primitive-cores 3 -i $my_png_file -o $my_output_folder

But I got the following error:

Error in parsing SVG: Unbound namespace prefix: "xlink"

Please help,

Thanks

Async I/O

It would be helpful if the blocking calls were removed and the API became async

Error when installing

Hey currently getting this error when trying to npm install everything

image

Is there a fix for this?

Vulnerability in js-yaml dependency

The js-yaml which is a dependency of multiple packages listed in package.json is reported to be a vulnerability.

Please check https://www.npmjs.com/advisories/813.

Steps to reproduce

  • Create a blank project.
- npm install --save sqip
- npm audit

Expected results

  • npm audit reports no vulnerabilities.

Actual results

npm audit reports a high severity vulnerability:

Screen Shot 2020-07-28 at 6 14 02 PM

Note:

The advisory mentions:

Versions of js-yaml prior to 3.13.1 are vulnerable to Code Injection.

I found something using~3.7.0 in the yarn.lock file, please check https://github.com/axe312ger/sqip/blob/master/yarn.lock#L8054

[QUESTION] Is this project dead?

I love this package! I've been using it for a while, but ran into an issue on Windows. In my opinion, this placeholder technique looks much better than any other alternative.

The issue has been fixed and merged in 206f683, however, the commit hasn't been published to npm... Which means without building and symlinking the package manually, this is unusable on Windows. Could we get another alpha version bump? Or should I look to package my own sqip binaries?

Sometimes the svg has different group structure

In some rare cases, primitive and/or svgo return a svg which is lacking a group.

This might not affect visual output but breaks the cli api › -o save result to file and basic svg structure is applied once in a while. Can be confusing especially for contributors

Example log:
https://circleci.com/gh/axe312ger/sqip/348

Research why and when this happens is very welcome. A PR to fix it as well :)

v1 umbrella issue

Alpha testers are very welcome, the README.md should be up to date for installation and usage instructions.

v1 will be released as soon all tickets & prs are closed in the v1 milestone.

Milestone:

https://github.com/axe312ger/sqip/milestone/1

Main features:

  • Async node api
  • Default output is 10%+ smaller gzipped as v0
  • Way more configuration options on CLI and Node API
  • Improved result metadata including color platte by node-vibrant
  • Improved CLI output and options
  • Plugin system
  • New demo
  • Improved test coverage
  • ...?

Core blockers

New LQIP alternative

Hey 👋

First off, I want to say that you folks have done an amazing job with sqip. The entire project is really impressive and I really loved the detailed comparison demo.

In the spirit of collaboration, I wanted to present a new take on LQIP which closely resembles Medium's approach: lqip-modern.

There are definitely tradeoffs with all of these different approaches, but for our use case, this approach provided the best combination of results in terms of visual fidelity, encoded image size, and encoding speed.

I also forked sqip's demo for lqip-modern's demo because I liked the setup so much.

Finally, you can see the completed LQIP effect in all its glory on this demo page: https://demo.notion2site.com/image-sizing-3492bd6dbaf44fe7a5cac62c5d402f06.

I'm creating this issue to start a discussion around a few areas:

  1. What do you think of this approach and the tradeoffs compared with sqip potrace, blurhash, and other LQIP techniques?

  2. Would you be open to a PR adding lqip-modern's default output to the sqip demo?

  3. I want to make sure that the benchmarks I'm presenting are correct and unbiased -- would love to get some feedback on this front. Specifically, lqip-modern is really only meant to be used blurred client-side, so this is something that I added to my fork of sqip's demo, but it's not currently blurring the non-lqip results. Thoughts here?

Thanks!

Hot Patch Current NPM release

Hi! I understand that you are hard at work with the next major revision of the of the project but, I was wondering if you could update Legacy branch to fix the js-yaml code injection warning. It would require bumping the dependency of svgo to 1.2.2

SVGO patch

CLI always outputs 300px-width svg

Hi,

I might be missing something obvious but when I'm running something like this...
npx sqip -i photo.jpg -n 40 -b 2 -m 1 -w 0 -o photo.svg
...the output is always 300px wide, instead of keeping the original image width.

Using a standard int (e.g. -w 1000) does not change this.

Updating the default CLI 300px width does not change this either.

{
name: 'width',
alias: 'w',
type: Number,
default: 300,
description:
'Width of the resulting file. Negative values and 0 will fall back to original image width.'
},

The only width that matters seems to be the one below. When changed (either to 0 or >0 value), the output width changes accordingly.

export default async function sqip(options) {
// Build configuration based on passed options and default options
const defaultOptions = {
plugins: [
{ name: 'primitive', options: { numberOfPrimitives: 8, mode: 0 } },
'blur',
'svgo',
'data-uri'
],
width: 300,
parseableOutput: false,
silent: true
}

Unable to find input file - NodeJS

I'm finding problems with sqip not finding files in my working directory

Actual Result
Changing the working directory from anything below, above or in the current working directory of sqip presents me with the error Error: Unable to find any files via /* . Make sure the file exists

Expected Result
File should be found in the current working directory, or any directory set as input.

Screenshot of code and directory structure:
Screen Shot 2019-12-26 at 2 40 05 AM

Output of error:

TypeError: resolver is not a function
    at Object.apiResolver (node_modules/next/dist/next-server/server/api-utils.js:39:9)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async DevServer.handleApiRequest (node_modules/next/dist/next-server/server/next-server.js:367:9)
    at async Object.fn /node_modules/next/dist/next-server/server/next-server.js:295:17)
    at async Router.execute (/node_modules/next/dist/next-server/server/router.js:25:32)
    at async DevServer.run (/node_modules/next/dist/next-server/server/next-server.js:417:29)
Something went wrong generating the SQIP previews
Error: Unable to find any files via /input.jpg. Make sure the file exists.

If you are using globbing patterns, the following features are supported:

https://github.com/micromatch/micromatch#matching-features
    at locateFiles (node_modules/sqip/dist/helpers.js:65:11)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async sqip (node_modules/sqip/dist/sqip.js:181:17)
    at async /.next/server/static/development/pages/api/generateImg.js:121:18

Reduce byte size further by improving SVG data URI encoding

Back when we first built SQIP, we went with Base64 encoding for the SVG string so that it was safe to be included inline in HTML because that is when SQIPs make their positive impact. See the "encodeBase64" function in the code.

However, back then, it was already clear to me that SVG, being XML, didn't strictly speaking need full base64 encoding and that we'd be making GZIP's / brotli's dictionaries slightly less successful in compression with the base64 overhead. Yet, no one had built a "proper" SVG data URI encoder at that time, so we went with base64 as a safe, if slightly bloated, bet.

Good news! Times have changed! There's now https://github.com/tigt/mini-svg-data-uri available, which is exactly the functionality we need and not a single byte more! Yay! 🎉 It will only encode the characters inside an SVG that are unsafe to be used in a data URI and leave the rest untouched.

With this library replacing the current base64 encoding, I'd expect that we can reduce the byte size of SQIPs by 3%-10%.

This improvement is very desirable in itself already to make SQIPs as small & fast as possible, but since SQIP byte size is already on par with raster-based LQIPs, we could even consider "reinvesting" these byte size savings into increasing the default shapes count of an uncustomized SQIP run by 1-2 shapes to further improve the visual accuracy of SQIPs.

create svgo plugin to reduce file size further more

SVGO does already a great job my minifying the result of primitive.

Rotated ellipses are output by primitive like this:

<ellipse cx="193" cy="143" fill="#441e0d" fill-opacity="0.501961" rx="41" ry="55"/>
<g transform="translate(152.621571 29.450687) rotate(272.983978) scale(14.437002 72.481412)">
  <ellipse cx="0" cy="0" fill="#ffb433" fill-opacity="0.501961" rx="1" ry="1"/>
</g>

While SVGO optimizes them to:

<g fill-opacity=".5" transform="translate(.6 .6) scale(1.17188)">
  <ellipse cx="193" cy="143" fill="#441e0d" rx="41" ry="55"/>
  <ellipse fill="#442010" rx="1" ry="1" transform="matrix(8.58762 -49.20482 40.32192 7.0373 194.7 138.8)"/>
</g>

There is no SVGO plugin yet which does reduce the precision of the matrix transform values.

A slight reduction might not harm our output at all while squeezing a few more bytes out of the file.

This should be a separate value from the svgo precision.

Reference SVG output

Primitive only, reformatted:

<svg xmlns="http://www.w3.org/2000/svg" width="300" height="187" version="1.1">
    <rect width="300" height="187" x="0" y="0" fill="#728d96"/>
    <g transform="scale(1.171875) translate(0.5 0.5)">
        <g transform="translate(152.621571 29.450687) rotate(272.983978) scale(14.437002 72.481412)">
            <ellipse cx="0" cy="0" fill="#ffb433" fill-opacity="0.501961" rx="1" ry="1"/>
        </g>
        <ellipse cx="193" cy="143" fill="#441e0d" fill-opacity="0.501961" rx="41" ry="55"/>
        <polygon fill="#fffacc" fill-opacity="0.501961" points="-16,140 178,136 59,175"/>
        <ellipse cx="214" cy="153" fill="#1d171c" fill-opacity="0.501961" rx="67" ry="12"/>
        <g transform="translate(75.252210 107.017801) rotate(358.449508) scale(107.341621 28.751537)">
            <ellipse cx="0" cy="0" fill="#2a7b96" fill-opacity="0.501961" rx="1" ry="1"/>
        </g>
        <g transform="translate(229.116868 107.779018) rotate(126.492013) scale(39.231272 21.501004)">
            <ellipse cx="0" cy="0" fill="#358aa6" fill-opacity="0.501961" rx="1" ry="1"/>
        </g>
        <ellipse cx="83" cy="62" fill="#7fb3d6" fill-opacity="0.501961" rx="253" ry="20"/>
        <g transform="translate(188.775422 111.992678) rotate(214.001902) scale(11.607972 27.554495)">
            <ellipse cx="0" cy="0" fill="#8a0000" fill-opacity="0.501961" rx="1" ry="1"/>
        </g>
    </g>
</svg>

Primitive + svgo reformatted:

<svg xmlns="http://www.w3.org/2000/svg" width="300" height="187">
    <defs/>
    <path fill="#728d96" d="M0 0h300v187H0z"/>
    <g fill-opacity=".5" transform="translate(.6 .6) scale(1.17188)">
        <ellipse cx="144" cy="30" fill="#ffba40" rx="70" ry="17"/>
        <ellipse fill="#442010" rx="1" ry="1" transform="matrix(8.58762 -49.20482 40.32192 7.0373 194.7 138.8)"/>
        <path fill="#fff6c7" d="M150.5 166.5H5.5v-29h145z"/>
        <ellipse fill="#217795" rx="1" ry="1" transform="matrix(1.16922 24.76311 -144.66103 6.83033 38.8 106.7)"/>
        <ellipse fill="#1d161a" rx="1" ry="1" transform="matrix(-76.48594 -5.06218 .92479 -13.97288 221.1 156.1)"/>
        <ellipse fill="#7ab2d7" rx="1" ry="1" transform="matrix(.96597 -19.75068 215.54079 10.5417 114.1 60)"/>
        <ellipse fill="#388ba4" rx="1" ry="1" transform="matrix(-31.23902 29.95715 -16.66446 -17.37754 234.7 108.2)"/>
        <ellipse cx="144" cy="29" fill="#ff9d54" rx="65" ry="14"/>
    </g>
</svg>

win32 installation problems

Hello. Firstly i want to say thanks for your work.
Today i tried to install sqip on my laptop with windows 10. And found few problems:

  1. child_process.execSync('type primitive') works incorrect for me (in checkForPrimitive function).
  2. process.env.PWD return undefined (in getInputfilePath function).

I fix that issues and create pull request (#14). Now all work good for me.
Please, check my solution. Maybe i do something wrong.
Thanks.

Next Version & Plugin system

As discussed via PM on twitter with @technopagan and @efegurkan, we should rewrite sqip to be a plugin based system.

This step could be used to refactor the codebase, here is my proposal:

Todo

  • transform the code into one style via eslint + prettier #31
  • replace argv with yargs. Allows more complex cli argument parsing. Replacing this and this #34
  • split up primitive svg group fix and blur filter to be used as or in plugins, remove unused xmldom and apply group fix via cheerio
  • transform api to async api and provide sync method #5 #42
  • upgrade all dependencies #42
  • add tests and aim for ~90% unit test coverage, make some integration/e2e tests. I'd go for jest and nixt
  • build the actual plugins #43 -->

Possible plugins

If you skip the plugins configuration, we just apply the current defaults to achieve simple blurred svg previews.

Current config:

const sqip = require('sqip');

const result =  sqip({
    filename: './input.jpg',
    numberOfPrimitives: 10,
    mode: 3,
    blur: 8
});

New plugin config:

const sqip = require('sqip');

const result =  sqip({
    filename: './input.jpg',
    plugins: [
      sqip.primitive({
        numberOfPrimitives: 10,
        mode: 3
        // just pass everything to primitive
      }),
      // Add blur if you need it
      sqip.blur({
        factor: 8
        // maybe allow other blur types?
      })
      // Blend in shapes over time (Would be my first plugin)
      sqip.loadingAnimation({
        ease: 'easeIn',
        duration: 3000
      })
    ]
});

The progress of this is tracked in the next branch. PR: #35

Output Image Not cropped to original dimensions

I have an input jpg image. It seems that the output svg encapsulates the extent of the primitives generated by primitive rather than cropping to the dimensions of the input image. Right now I have to do that manually in illustrator.

document what the node api & cli return

node

{
  svg: '<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 300 188">...</svg>',
  metadata: {
    originalWidth: 1024,
    originalHeight: 640,
    palette: {
      // https://github.com/akfish/node-vibrant#vibrantswatch
      Vibrant: [Swatch],
      DarkVibrant: [Swatch],
      LightVibrant: [Swatch],
      Muted: [Swatch],
      DarkMuted: [Swatch],
      LightMuted: [Swatch]
    },
    width: 300,
    height: 188,
    type: 'svg',
    dataURI: "data:image/svg+xml,...",
    dataURIBase64: 'data:image/svg+xml;base64,...'
  }
}

CLI

Fully featured

$ sqip -i __tests__/fixtures/beach.jpg
Processing: __tests__/fixtures/beach.jpg
[Preview image (iTerm2 users only)]
┌───────────────┬────────────────┬───────┬────────┬──────┐
│ originalWidth │ originalHeight │ width │ height │ type │
├───────────────┼────────────────┼───────┼────────┼──────┤
│ 1024          │ 640            │ 300   │ 188    │ svg  │
└───────────────┴────────────────┴───────┴────────┴──────┘
┌─────────┬─────────────┬──────────────┬─────────┬───────────┬────────────┐
│ Vibrant │ DarkVibrant │ LightVibrant │ Muted   │ DarkMuted │ LightMuted │
├─────────┼─────────────┼──────────────┼─────────┼───────────┼────────────┤
│ #dd852f │ #be4e0c     │ #f2b17a      │ #5c8fa4 │ #694e35   │ #cfc8b7    │
└─────────┴─────────────┴──────────────┴─────────┴───────────┴────────────┘

non-TTY & --parseable-output input flag

$ sqip -i __tests__/fixtures/beach.jpg --parseable-output
Processing: __tests__/fixtures/beach.jpg
originalWidth originalHeight width height type
1024          640            300   188    svg
Vibrant DarkVibrant LightVibrant Muted   DarkMuted LightMuted
#dd852f #be4e0c     #f2b17a      #5c8fa4 #694e35   #cfc8b7

--silent flag

$ sqip -i __tests__/fixtures/beach.jpg --silent

SQIP generation blocks Node

I absolutely love this package, but I can't solve a huge problem with it.

It completly blocks my server while processing the image!
I was hoping to fix this by using separate job (using Bull queue) but no success. Whole server stops and doesn't respond to requests when this job is started. Job is started in separate thread, but still SQIP completely blocks everything!

How do I handle this? Please help me guys, really want to use this is production.

UPD: after all I managed to process SQIP in separate process. Now it doesn't block server.

Option for outputting the raw svg

To enable combinatory use with other tool chains it would be beneficial to be able to output just the raw svg without an <img>-tag.

Support configuration options for underlying SVG converter

Is there any plan to support further configuration of the underlying SVG converter?
In the current version primitive is used to create the SVGs and according to the documentation there are more command line flags available than just the number of shapes, which is configurable by this library.
Our current use case would be the usage of the -bg argument, which can be used to provide a specific background color. We experienced issues with png files with transparent backgrounds, which turn out to be black instead of transparent when converted.
Additionally also the -m argument would be of interest for further experimentation.
I can offer to provide you with a pull request when there is nothing like that currently in the works.. 👍

SQIP generates image with transparent line

Hello, and first of thanks for absolute awesome package. Really incredible!

After playing around I noticed that output contains unwanted transparent space. How to get rid of it?

Here you can find it on the bottom (whole bottom line)

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1024 647"><path fill="#91816b" d="M0 0h1024v644H0z"/><g transform="matrix(4 0 0 4 2 2)" fill-opacity=".5"><ellipse rx="1" ry="1" transform="matrix(27.4936 -2.70998 8.04994 81.66923 191.1 55.7)"/><path fill="#fff" d="M266 159l-66 5 57-115z"/><ellipse fill="#ace2b3" cx="25" cy="117" rx="59" ry="59"/><ellipse cx="198" cy="103" rx="33" ry="27"/><ellipse fill="#f48051" rx="1" ry="1" transform="matrix(-16.80109 50.21316 -24.47792 -8.1902 150.8 72.5)"/><ellipse rx="1" ry="1" transform="matrix(-46.1538 -80.82873 15.12055 -8.63395 171.2 0)"/><ellipse fill="#e5ffcf" rx="1" ry="1" transform="matrix(-17.67696 2.01953 -6.85804 -60.0286 243.8 84.7)"/><path fill="#ffa46c" d="M159.4 68.1l13.9-17 23.3 18.8-13.9 17z"/><ellipse fill="#6d1730" rx="1" ry="1" transform="matrix(3.789 -24.1097 16.88316 2.6533 97.3 98.3)"/><path fill="#1f090d" d="M185.1-16l-64.6 35.5L168.8 41l-44.4-1.3z"/></g></svg>

Sorry, Github doesn't support SVG

dyld: lazy symbol binding failed: Symbol not found: _g_once_impl - Update Sharp Dependency

First time reporter, but long time user. I really like your package and it's Primitive SVG generator. I've used it in my project to process all images I've uploaded to generate placeholders that are loaded while images are loading.

Currently I'm facing an issue with sharp within your package being outdated.

dyld: lazy symbol binding failed: Symbol not found: _g_once_impl
 Referenced from: /Users/thomas/Documents/Github/bff/node_modules/plugin-upload/node_modules/sharp/build/Release/sharp.node
  Expected in: /Users/thomas/Documents/Github/bff/node_modules/sqip/node_modules/sharp/build/Release/../../vendor/lib/libvips-cpp.42.dylib

Doing some background research it's referenced there's a conflict packages using outdated Sharps. I'm currently using the solution provided in the link below, but also see you've been working on updating the package to TS in a recent branch.

If it's also possible for you to update the dependencies used in your package, It'd be greatly appreciated. I'd like to continue using it without conflict.

22mahmoud/eleventy-image-example#1

Thanks for creating this awesome package. 👍🏼

Preserve white background

Hi, I have a series of product images (mainly beer bottles and cans) that consist in the product and a white background. When processed through sqip, the background always end up in some light gray color, making the loading effect less smooth.

Is there any way to preserve the white background in the svg? (either by preprocessing with imagemagick/graphicsmagick or with any other css trick) Thanks in advance!

Pixelated effect

Hello. I just thought it would be cool to have pixelated effect. How do you think it's possible?

Publish SQIP on NPM

Let's publish SQIP on NPM to make the installation much easier for everyone.

error when reading certain jpg files as input

Hey,

I've started to use recently the sqip library, the alpha version and is working great.

Lately, for some images, I'm getting this error

TypeError: Corrupt JPG, exceeded buffer limits
at validateBuffer (/node_modules/image-size/lib/types/jpg.js:98:11)
at Object.calculate (/node_modules/image-size/lib/types/jpg.js:122:5)
at lookup (/node_modules/image-size/lib/index.js:26:35)
at module.exports (/node_modules/image-size/lib/index.js:122:12)
at processImage (/node_modules/sqip/dist/sqip.js:93:48)
at sqip (/node_modules/sqip/dist/sqip.js:199:26)

which seems to be caused by the image-size library, I've found more details about this topic here image-size/image-size#37

It seems that by switching to https://github.com/nodeca/probe-image-size the issue is no longer present.

Thanks a lot for your work!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.