open-dating / adult-image-detector Goto Github PK
View Code? Open in Web Editor NEWUse deep neural networks and other algos for detect nude images
Home Page: https://adult-image-detector.herokuapp.com/
Use deep neural networks and other algos for detect nude images
Home Page: https://adult-image-detector.herokuapp.com/
hi,
as of now, raw images are supported. I would like to add support for extracting images from PDF file and detect.
If you think this is a good addition, I can send a PR.
Is there any way how to upload my file via url?
For example:
curl -i -X POST -F "image=@http://cdn.akc.org/content/article-body-image/samoyed_puppy_dog_pictures.jpg" http://localhost:9191/api/v1/detect
i can't find the output images
I got this error when build the docker
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 3.01kB 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 45B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:18.04 0.0s
=> [internal] load build context 0.2s
=> => transferring context: 24.13MB 0.2s
=> [opencv 1/3] FROM docker.io/library/ubuntu:18.04 0.0s
=> [opencv 2/3] RUN apt-get update && apt-get install -y --no-install-recommends git build-essential cmake pkg-config unzip libgtk2.0-dev curl ca-certificates libcurl4-openssl-dev libssl-dev libavcodec-dev libavformat-dev libswscale-dev libtbb2 libtbb-dev 155.9s
=> [opencv 3/3] RUN curl -Lo opencv.zip https://github.com/opencv/opencv/archive/4.5.1.zip && unzip -q opencv.zip && curl -Lo opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.5.1.zip && unzip -q opencv_contrib.zip && rm opencv.zip o 292.2s
=> [gocv 1/3] RUN apt-get update && apt-get install -y --no-install-recommends git software-properties-common && curl -Lo go1.14.2.linux-amd64.tar.gz https://dl.google.com/go/go1.14.2.linux-amd64.tar.gz && tar -C /usr/local -xzf go1.14.2.linux-amd64.tar.gz && 39.8s
=> [gocv 2/3] RUN mkdir -p "/go/src" "/go/bin" && chmod -R 777 "/go" 0.4s
=> [gocv 3/3] WORKDIR /go 0.0s
=> ERROR [adult-image-detector 1/5] RUN go get -u github.com/pilu/fresh 11.9s
------
> [adult-image-detector 1/5] RUN go get -u github.com/pilu/fresh:
#0 11.86 # golang.org/x/sys/unix
#0 11.86 src/golang.org/x/sys/unix/syscall.go:83:16: undefined: unsafe.Slice
#0 11.86 src/golang.org/x/sys/unix/syscall_linux.go:2271:9: undefined: unsafe.Slice
#0 11.86 src/golang.org/x/sys/unix/syscall_unix.go:118:7: undefined: unsafe.Slice
#0 11.86 src/golang.org/x/sys/unix/sysvshm_unix.go:33:7: undefined: unsafe.Slice
------
Dockerfile:64
--------------------
62 | FROM gocv AS adult-image-detector
63 | # ENV GO111MODULE onγ
64 | >>> RUN go get -u github.com/pilu/fresh
65 |
66 | WORKDIR $GOPATH/src/adult-image-detector
--------------------
ERROR: failed to solve: process "/bin/sh -c go get -u github.com/pilu/fresh" did not complete successfully: exit code: 2
how I train and teach this for detect perfectly?
Hello!
First, thanks for nice project, I like it, especially docker version, and I'm using it in my Nude Crawler project.
Problem: when using docker image it almost never releases memory, and takes more and more (until will get our of memory).
How to reproduce problem:
# start and see how much memory it uses (tiny 11Mb)
$ sudo docker run -d --rm -p 9191:9191 --name aid --memory=1G opendating/adult-image-detector
$ sudo docker stats --no-stream
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
6b22f14d33a3 aid 0.00% 11.27MiB / 1GiB 1.10% 5.11kB / 0B 11.6MB / 0B 6
# Analyse first file
$ curl -s -i -X POST -F "image=@/tmp/eropicture.jpg" http://localhost:9191/api/v1/detect > /dev/null
$ sudo docker stats --no-stream
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
6b22f14d33a3 aid 0.00% 170MiB / 1GiB 16.60% 315kB / 1.9kB 35.9MB / 0B 10
# and two more
$ curl -s -i -X POST -F "image=@/tmp/eropicture.jpg" http://localhost:9191/api/v1/detect > /dev/null
$ curl -s -i -X POST -F "image=@/tmp/eropicture.jpg" http://localhost:9191/api/v1/detect > /dev/null
# Why now it needs extra 50Mb after two more images analysed?? Ok...
$ sudo docker stats --no-stream
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
6b22f14d33a3 aid 0.00% 221.7MiB / 1GiB 21.65% 932kB / 5.48kB 35.9MB / 0B 10
# Lets analyse 100 more images
$ for i in `seq 1 100`; do curl -s -i -X POST -F "image=@/tmp/eropicture.jpg" http://localhost:9191/api/v1/detect > /dev/null ; done
# now 500Mb used for something...
$ sudo docker stats --no-stream
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
6b22f14d33a3 aid 0.00% 545.4MiB / 1GiB 53.26% 31.7MB / 174kB 36MB / 614kB 11
# Now it uses about all of available limit, sometimes 1022MiB, sometimes falls back to 976MiB,
# but about at 700-800 curl request container crashes.
# If run in foreground, no error messages displayed, last messages are:
2023/03/17 18:34:21 For file 2023-03-17T18:34:21Z_fabede53-3817-47f8-a077-86d712d51602.jpg, openNsfwScore=0.834231
2023/03/17 18:34:21 For file 2023-03-17T18:34:21Z_fabede53-3817-47f8-a077-86d712d51602.jpg, anAlgorithmForNudityDetection=true
2023/03/17 18:34:21 Uploaded file eropicture.jpg, saved as 2023-03-17T18:34:21Z_3c3370b3-ede2-4847-ac04-9ba5957765b0.jpg
Test image size is 299K. All tests I did on docker image I pulled today.
If set RAM limit to 300Mb, it goes to almost full memory much sooner, at about 20-30th request, but continues to work at this memory usage up to ~100th request. (then crashes). When I start it without memory limit, it runs much longer (I have 16Gb RAM), but in the and machine hits OOM anyway.
If stop sending new requests, it keeps almost full memory for long time (I tried to wait for few hours, memory usage is nearly same after few hours of idle time).
In theory, I could restart docker container after every N requests, but it looks ugly. And my script runs as ordinary user account, I would not like to give it root access even to analyze nude woman.
Maybe it's possible to tune up garbage collector or restart process? Or maybe make API call (which I would trigger after every N requests) which will release memory or restart program?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.