tg-bomze/face-depixelizer

Face Depixelizer based on "PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models" repository.

NOTE

We have noticed a lot of concern that PULSE will be used to identify individuals whose faces have been blurred out. We want to emphasize that this is impossible - PULSE makes imaginary faces of people who do not exist, which should not be confused for real people. It will not help identify or reconstruct the original image.

We also want to address concerns of bias in PULSE. We have now included a new section in the paper and an accompanying model card directly addressing this bias.

If you are interested more about the topic, you can read this IEEE Tech Talk about PULSE.

Face-Depixelizer

Face Depixelizer based on "PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models" repository.

Given a low-resolution input image, Face Depixelizer searches the outputs of a generative model (here, StyleGAN) for high-resolution images that are perceptually realistic and downscale correctly.

Check how it works on Google Colab:

• Russian Language
• English Language

Based on: PULSE

Article: PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

Currently using Google Drive to store model weights and it has a daily cap on downloads, therefore, you may receive an error message saying "Google Drive Quota Exceeded" or "No such file or directory: '/content/pulse/runs/face.png'". If you are experiencing this error please try again later in the day or come back tomorrow.

Thanks for the help in fixing the errors: AlrasheedA, kuanhulio, DevMentor

face-depixelizer's Issues

keep getting the error name "toPIL" is not defined

it says it in the title, keep using it and i keep getting this error

train your own dataset

hi, can you please explain how can I train my own dataset on Face-depixelizer or on pulse

I am trying to test it on some of my own images which are very low resolution. I thought it may be a good idea to train pulse on my own dataset. Any suggestions are appreciated.

Could not find a face that downscales correctly within epsilon

i keep getting that message even if my uploaded is square - i get the message whether the photo is unretouched, or is pixelated via photoshop pixelate effect option. what should the input be - unretouched? or pixelated? i was also careful about fitting my head inside the outline but it keeps returning with above message

OSError: Google Drive quota exceeded

For three days now, I have not been able to get it to work on Colab.
Every time, I upload an image, that succeeds, then it tries to download something from a Google Drive link and I get this output:

Estimated Runtime: 29s.

Loading Synthesis Network
Downloading https://drive.google.com/uc?id=1TCViX1YpQyRsklTVYEJwdbmK91vklCo8 ............ failed
Traceback (most recent call last):
  File "run.py", line 58, in <module>
    model = PULSE(cache_dir=kwargs["cache_dir"])
  File "/content/pulse/PULSE.py", line 23, in __init__
    with open_url("https://drive.google.com/uc?id=1TCViX1YpQyRsklTVYEJwdbmK91vklCo8", cache_dir=cache_dir, verbose=verbose) as f:
  File "/content/pulse/drive.py", line 66, in open_url
    raise IOError("Google Drive quota exceeded")
OSError: Google Drive quota exceeded
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-1-af0d780e3ec8> in <module>()
     90 ax1.imshow(mpimg.imread('/content/pulse/input/face.png'))
     91 ax1.set_title('Original')
---> 92 ax2.imshow(mpimg.imread('/content/pulse/runs/face.png'))
     93 ax2.set_title('Result')
     94 plt.show()

3 frames
/usr/local/lib/python3.6/dist-packages/matplotlib/cbook/__init__.py in to_filehandle(fname, flag, return_opened, encoding)
    401             fh = bz2.BZ2File(fname, flag)
    402         else:
--> 403             fh = open(fname, flag, encoding=encoding)
    404         opened = True
    405     elif hasattr(fname, 'seek'):

FileNotFoundError: [Errno 2] No such file or directory: '/content/pulse/runs/face.png'

I would really love to try it out, but sadly I can't. Could you look into that please?

Add strings to use custom own user's GoogleDrive to manipulate with images

to fix this issue: #14

HTTPError: 403

Estimated Runtime: 29s.

Loading Synthesis Network
Downloading https://drive.google.com/uc?id=1TCViX1YpQyRsklTVYEJwdbmK91vklCo8 ............ failed
Traceback (most recent call last):
File "run.py", line 58, in
model = PULSE(cache_dir=kwargs["cache_dir"])
File "/content/pulse/PULSE.py", line 23, in init
with open_url("https://drive.google.com/uc?id=1TCViX1YpQyRsklTVYEJwdbmK91vklCo8", cache_dir=cache_dir, verbose=verbose) as f:
File "/content/pulse/drive.py", line 54, in open_url
res.raise_for_status()
File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://doc-04-1s-docs.googleusercontent.com/docs/securesc/r9ofj9camu6bj09nouave131nqpgs994/jbcg5uh6k4l5f1mm6cb005euurd27fq7/1592851875000/08253799648200435663/11878159376741077572Z/1TCViX1YpQyRsklTVYEJwdbmK91vklCo8

---

FileNotFoundError Traceback (most recent call last)

in ()
90 ax1.imshow(mpimg.imread('/content/pulse/input/face.png'))
91 ax1.set_title('Original')
---> 92 ax2.imshow(mpimg.imread('/content/pulse/runs/face.png'))
93 ax2.set_title('Result')
94 plt.show()

keep getting this error

NameError Traceback (most recent call last)
in ()
1 #@title ← Download result
----> 2 try: files.download('/content/pulse/runs/face.png')
3 except: raise Exception("No result image")

NameError: name 'files' is not defined

During handling of the above exception, another exception occurred:

Exception Traceback (most recent call last)
in ()
1 #@title ← Download result
2 try: files.download('/content/pulse/runs/face.png')
----> 3 except: raise Exception("No result image")

Exception: No result image

When input cannot be processed, it just displays last output

E.g., Could not find a face that downscales correctly within epsilon shouldn't show the side-by-side of the new input and the last successful output. It should just show no output.

Run 1:

Run 2:

Run 3:

Could not find a face that downscales correctly within epsilon

Hello,

I am running the colab from my google drive after uploading synthesis.pt, mapping.pt and shape_predictor_68_face_landmarks.dat and changing the respective paths. The code is executing without error, but the faces produced do not seam to match others' results with the default settings and I think it has to do with the "Could not find a face that downscales correctly within epsilon" text the code is producing with the results.

Here is what I am getting vs. what I think should be the result based on https://twitter.com/tg_bomze/status/1274245778551328769/photo/1

Any help would be appreciated, and thank you for providing such great and fun code!

Ethical considerations

In the current climate of mass law enforcement and government surveillance plus mass protest in the United States, I find the release of this technology disturbing.

I urge you to read the core 4 principles of the Algorithmic Justice League:

• Affirmative consent: Everyone should have a real choice in how and whether they interact with AI systems.
• Meaningful transparency: It is of vital public interest that people are able to understand the processes of creating and deploying AI in a meaningful way, and that we have full understanding of what AI can and cannot do.
• Continuous oversight and accountability: Politicians and policymakers need to create robust mechanisms that protect people from the harms of AI and related systems both by continuously monitoring and limiting the worst abuses and holding companies and other institutions accountable when harms occur. Everyone, especially those who are most impacted, must have access to redress from AI harms. Moreover, institutions and decision makers that utilize AI technologies must be subject to accountability that goes beyond self-regulation.
• Actionable critique: We aim to end harmful practices in AI, rather than name and shame. We do this by conducting research and translating what we’ve learned into principles, best practices and recommendations that we use as the basis for our advocacy, education and awareness-building efforts. We are focused on shifting industry practices among those creating and commercializing today’s systems.

How does this work interact with those principles? We can see what this technology does, but what is its intended use? What are the worst-case scenarios for the use of this openly-available technology around the world? How will further technology built on top of this work be used?

PULSE does not contain a publicly available license to determine what type of use is acceptable or not. I urge you to consider licensing this derivative work under stipulations that require any end users to Do No Harm, like the Hippocratic License.

(In case there is any ambiguity, I am speaking as an individual in the technology industry, not on behalf of or with any relation to my employer.)

OSError: Google Drive quota exceeded

always shows this error even though i keep trying at different times

TypeError: google.colab._files is undefined

I'm not very familiar with Python, so sorry for my ignorance here, and inability to explain what's probably going wrong. Running the Google Colab notebook without changing any settings causes the following error to appear...

Upload widget is only available when the cell has been executed in the current browser session. Please rerun this cell to enable.

---------------------------------------------------------------------------

MessageError                              Traceback (most recent call last)

<ipython-input-1-5483e66a8121> in <module>()
     46   get_ipython().system('rm -rf /content/pulse/input/face.png')
     47   clear_output()
---> 48   uploaded = files.upload()
     49   for fn in uploaded.keys():
     50     print('User uploaded file "{name}" with length {length} bytes'.format(

2 frames

/usr/local/lib/python3.6/dist-packages/google/colab/files.py in upload()
     62   result = _output.eval_js(
     63       'google.colab._files._uploadFiles("{input_id}", "{output_id}")'.format(
---> 64           input_id=input_id, output_id=output_id))
     65   files = _collections.defaultdict(_six.binary_type)
     66   # Mapping from original filename to filename as saved locally.

/usr/local/lib/python3.6/dist-packages/google/colab/output/_js.py in eval_js(script, ignore_result)
     37   if ignore_result:
     38     return
---> 39   return _message.read_reply_from_input(request_id)
     40 
     41 

/usr/local/lib/python3.6/dist-packages/google/colab/_message.py in read_reply_from_input(message_id, timeout_sec)
    104         reply.get('colab_msg_id') == message_id):
    105       if 'error' in reply:
--> 106         raise MessageError(reply['error'])
    107       return reply.get('data', None)
    108 

MessageError: TypeError: google.colab._files is undefined

Pulse on video

@tg-bomze @kuanhulio

Can pulse be trained on video, or does it only work on faces?

@kuanhulio i have sent you an email, have a look

Max stack call exceeded

On subsequent runs (only), immediately after uploading a picture, the script errors out as follows:

 ---------------------------------------------------------------------------
MessageError                              Traceback (most recent call last)
<ipython-input-5-5483e66a8121> in <module>()
     46   get_ipython().system('rm -rf /content/pulse/input/face.png')
     47   clear_output()
---> 48   uploaded = files.upload()
     49   for fn in uploaded.keys():
     50     print('User uploaded file "{name}" with length {length} bytes'.format(

2 frames
/usr/local/lib/python3.6/dist-packages/google/colab/_message.py in read_reply_from_input(message_id, timeout_sec)
    104         reply.get('colab_msg_id') == message_id):
    105       if 'error' in reply:
--> 106         raise MessageError(reply['error'])
    107       return reply.get('data', None)
    108 

MessageError: RangeError: Maximum call stack size exceeded.

I can't tell what the problem is but it could be a failure to clean up after a failed run.

Documentation and code for training

@tg-bomze @kuanhulio

It is vital to have documentation on how to train our own model.
prepare the training data

Doesn't work – confusing instructions

How to use Face Depixelizer properly? The instructions tell me to click on the "Let's Rock!" button but every time I get the error:

MessageError                              Traceback (most recent call last)
<ipython-input-7-af0d780e3ec8> in <module>()
     46   get_ipython().system('rm -rf /content/pulse/input/face.png')
     47   clear_output()
---> 48   uploaded = files.upload()
     49   for fn in uploaded.keys():
     50     print('User uploaded file "{name}" with length {length} bytes'.format(

2 frames
/usr/local/lib/python3.6/dist-packages/google/colab/_message.py in read_reply_from_input(message_id, timeout_sec)
    104         reply.get('colab_msg_id') == message_id):
    105       if 'error' in reply:
--> 106         raise MessageError(reply['error'])
    107       return reply.get('data', None)
    108 

MessageError: TypeError: Cannot read property '_uploadFiles' of undefined

Why is the "Choose a file" button deactivated? Is it a problem regarding the daily limit?

Always get "Could not find a face that downscales correctly within epsilon"

Thank you very much for the nice repository. I tried several pics, but I always get this error:
'''
Estimated Runtime: 29s.

Loading Synthesis Network
Optimizing
BEST (100) | L2: 0.0053 | GEOCROSS: 3.1139 | TOTAL: 0.6826 | time: 57.3 | it/s: 1.75 | batchsize: 1
Could not find a face that downscales correctly within epsilon

[Errno 2] No such file or directory: '/content/pulse/runs/face.png'
'''

Could you please point out a direction which I could work on to solve this problem?

Possibility to run locally to avoid Drive quota being exceeded?

Is there a way to run this locally or to point it to, say, one's own Google Drive to avoid the issue of qouta being exceeded?

FileNotFoundError

I'm getting this error while running project on colab :
FileNotFoundError: [Errno 2] No such file or directory: '/content/pulse/runs/face.png'

RuntimeError when uploading image

When I upload this image (32x32 .png) using english or russian colab, I get the following error:

mario.png(image/png) - 2858 bytes, last modified: n/a - 100% done

Saving mario.png to mario.png
User uploaded file "mario.png" with length 2858 bytes

---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

<ipython-input-1-50b27c8aa454> in <module>()
     64   imagesHR = []
     65   imagesHR.append(face)
---> 66   face = toPIL(D(toTensor(face).unsqueeze(0).cuda()).cpu().detach().clamp(0,1)[0])
     67   images.append(face)
     68 

1 frames

/content/pulse/bicubic.py in forward(self, x, nhwc, clip_round, byte_output)
     59         # downscaling performed by 1-d convolution
     60         x = F.pad(x, (0, 0, pad_top, pad_bottom), self.padding)
---> 61         x = F.conv2d(input=x, weight=filters1, stride=(stride, 1), groups=3)
     62         if clip_round:
     63             x = torch.clamp(torch.round(x), 0.0, 255.)

RuntimeError: Given groups=3, weight of size [3, 1, 4, 1], expected input[1, 4, 1027, 1024] to have 3 channels, but got 4 channels instead

Error after pressing "let's rock!"

When i press "Let's rock!" i get this error
MessageError Traceback (most recent call last)
in ()
46 get_ipython().system('rm -rf /content/pulse/input/face.png')
47 clear_output()
---> 48 uploaded = files.upload()
49 for fn in uploaded.keys():
50 print('User uploaded file "{name}" with length {length} bytes'.format(

2 frames
/usr/local/lib/python3.7/dist-packages/google/colab/_message.py in read_reply_from_input(message_id, timeout_sec)
104 reply.get('colab_msg_id') == message_id):
105 if 'error' in reply:
--> 106 raise MessageError(reply['error'])
107 return reply.get('data', None)
108

MessageError: TypeError: Cannot read properties of undefined (reading '_uploadFiles')

I resetted the runtime and tried over and over again pressing the button. i always get this error. what should i do?