lariffle/ariann

Low-Interaction Privacy-Preserving Deep Learning via Function Secret Sharing

ariann's Issues

torchscprng corresponding to torch=1.4.0?

Hi, in the ryffel/ariaNN branch of OpenMined/PySyft ryffel/0.2.x-fix-training, we need import torchscprng, but torchscprng don't support torch==1.4.0. However, we need torch==1.4.0 for syft==0.2.9, how to resolve this problem? Thanks.

no ryffel/ariaNN branch in OpenMined/PySyft？

Hi, I was trying to run the code by installing from the source. but there are some errors when running the code below：

>>> cd ~
>>> git clone https://github.com/OpenMined/PySyft.git
>>> cd PySyft
>>> git checkout a73b13aa84a8a9ad0923d87ff1b6c8c2facdeaa6       ## some exceptions
Note: checking out 'a73b13aa84a8a9ad0923d87ff1b6c8c2facdeaa6'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at a73b13a... Remove softmax logs
>>> pip install -e .                   ## there is sone errors here
...

Also, I found out that there is no ryffel/ariaNN branch in OpenMined/PySyft, I think this is the crux of the problem, is that the case?

Amortized results in the paper

Hi, just to be clear that the amortized results means if say we measure the inference of a 128 batch, the final results of time and comm. are divided by 128 right?

Communication rounds data

Hi there! Brilliant work from your team!

One question I have when testing: the comm_info flag does not give the total communication rounds online, which to me is a major improvement of your method. Is there any way to count that? Thanks!

Error when running WAN setting experiment

Hi! I was trying to run the WAN setting by installing from the source. I followed all instructions except two things: 1. I created a python venv for the project and 2. I put all folders into ~/work instead of in ~. I've changed the home dir in the .sh files accordingly.

Without adding the --websocket flag, the experiment runs normally. However if with it, I got the following error saying "Connection refused".

Do I have to do something in advance to initialize the port? The firewall on the server should be closed.

The PySyft version.

Hi, what are the versions of syft, syft-proto, torch and torchvision? The latest PySyft does not support your project.

How to view the detailed implementation of FSS ?

Happy New Year, Ryffel～

Currently I am studying Function Secret Sharing(FSS) and interested in the implementation of FSS, so I would like to know how FSS is implemented in AriaNN.

In AriaNN, PySyft is used. And then I studied the FSS implementation in PySyft, but in some branches (I used PySyft-ryffel-0.2.x-fix-training), FSS is implemented based on the sycret. However, the sycret in AriaNN does not give a specific implementation, the details are included in Dynamic link library (e.g. native.so) and cannot be viewed, I hope you can give some more information so that I can understand how to implement FSS.

To better describe the problem, here is an example:

import torch
import torch.nn.functional as F
import syft as sy

hook = sy.TorchHook(torch)
bob = sy.VirtualWorker(hook, id="bob")
alice = sy.VirtualWorker(hook, id="alice")
crypto_provider = sy.VirtualWorker(hook, id="crypto_provider")

X = torch.rand(128, 128)
X_ = X.fix_precision(precision_fractional = 4).share(bob, alice, crypto_provider=crypto_provider, protocol="fss")

Y_ = F.relu(X_)         # call relu
Y = Y_.get()

In order to calculate the relu function, class DIF will be called, and it is defined in PySyft/syft/frameworks/torch/mpc/fss.py:

class DIF:
    """Distributed Interval Function - used for comparison"""
    @staticmethod
    def keygen(n_values=1):
        return dif.keygen(n_values=n_values)    # dif = sycret.LeFactory(n_threads=N_CORES)

    @staticmethod
    def eval(b, x, k_b):
        # x = x.astype(np.uint64)
        original_shape = x.shape
        x = x.reshape(-1)
        flat_result = dif.eval(b, x, k_b)    # dif = sycret.LeFactory(n_threads=N_CORES)
        return flat_result.astype(np.int32).astype(np.int64).reshape(original_shape)

It can be seen that the relu function is based on the sycret which is located in the library of the current virtual python environment /root/.../anconda3/envs/pysyft/lib/python3.7/site -packages/sycret. In sycret, it can be seen from from .sycret import lib and lib.keygen, lib.eval that the specific FSS function is defined in lib, but lib cannot be viewed.

which python

Hi,

Could you please advise which version of python you were using in this repo?
Thanks in advance!

Training Network2 with --public argument.

Hi! I'm attempting to test out the code with certain parameters and I've noticed that it is impossible to train the Network2 model when the --public argument is passed. Is there a specific reason for this?

Traceback:

$ python main.py --model network2 --dataset mnist --train --public --epochs 5
Training over 5 epochs
model:           network2
dataset:         mnist
batch_size:      128
Traceback (most recent call last):
  File "main.py", line 314, in <module>
    run(args)
  File "main.py", line 91, in run
    train_time = train(args, model, private_train_loader, optimizer, epoch)
  File "~/new_test/ariann/procedure.py", line 61, in train
    loss[0] = forward(optimizer, model, data, target)
  File "~/new_test/ariann/procedure.py", line 50, in forward
    loss_enc = output.cross_entropy(target)
  File "~/PySyft/syft/generic/frameworks/hook/hook.py", line 162, in overloaded_native_method
    method = getattr(self, f"native_{method_name}")
AttributeError: 'Tensor' object has no attribute 'native_cross_entropy'

Training on other models such as LeNet or Network1, works correctly, with and without the --public argument. Training on Network2 works if the --public argument is not used.

The problem seems to be how the CrossEntropyLoss is called with PySyft -- namely, it fails to get the correct attribute from PyTorch, if the values are unencrypted/secret shared. After attempting to change the hook.py code to use 'cross_entropy' instead of 'native_cross_entropy' it gets stuck in a loop and causes a RecursionError:

  File "~/PySyft/syft/generic/frameworks/hook/hook.py", line 174, in overloaded_native_method
    raise route_method_exception(e, self, args, kwargs)
  File "~/PySyft/syft/generic/frameworks/hook/hook.py", line 170, in overloaded_native_method
    response = method(*args, **kwargs)
RecursionError: maximum recursion depth exceeded

Thank you for any information or insight you could provide!

Communication for training

Hi,

I am trying to get the total communication overhead when training the neural networks. If I am not wrong, the --comm_info flag only prints out the total communication per item in the test function, but not for the train function, right?