jeshraghian / snntorch Goto Github PK
View Code? Open in Web Editor NEWDeep and online learning with spiking neural networks in Python
Home Page: https://snntorch.readthedocs.io/en/latest/
License: MIT License
Deep and online learning with spiking neural networks in Python
Home Page: https://snntorch.readthedocs.io/en/latest/
License: MIT License
The .flake8
config currently excludes lots of hints and errors. The code-base should be cleaned with the standard Flake8 config.
I would be glad to help out. Reporting as a reminder.
I've been trying to train a model in an IPU environment using PopTorch and snntorch-ipu. Unfortunately, I always get a crash. It is unclear to me what exactly is going on, so hopefully someone knows.
If I try to train my model with only snntorch-ipu installed, as recommended, I will always get an error message when importing/working with surrogates about "Missing Straight Through Estimator Custom Operation file".
/notebooks/dvsclf/network/net.py in <module>
1 import torch
----> 2 from snntorch import surrogate
3 from snntorch import utils
4 import torch.nn as nn
5 import numpy as np
/usr/local/lib/python3.8/dist-packages/snntorch/surrogate.py in <module>
26
27
---> 28 class StraightThroughEstimator:
29 """
30 Straight Through Estimator.
/usr/local/lib/python3.8/dist-packages/snntorch/surrogate.py in StraightThroughEstimator()
53 print("Missing Straight Through Estimator Custom Operation file!")
54 print(so_path_ste)
---> 55 exit(1)
56 ctypes.cdll.LoadLibrary(so_path_ste)
57
NameError: name 'exit' is not defined
If I install snntorch (with or without snntorch-ipu beside it), I will not get the above error. Instead, something in PopTorch throws an error when the model is being trained.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-4-9ee252bfabe9> in <module>
2 # Performs forward pass, loss function evaluation,
3 # backward pass and weight update in one go on the device.
----> 4 _, loss = poptorch_model(batch, target)
[....]
/notebooks/dvsclf/network/snn.py in forward(self, x)
20
21 x = self.conv(x)
---> 22 x = self.lif(x)
23 return x
24
/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1118 input = bw_hook.setup_input_hook(input)
1119
-> 1120 result = forward_call(*input, **kwargs)
1121 if _global_forward_hooks or self._forward_hooks:
1122 for hook in (*_global_forward_hooks.values(), *self._forward_hooks.values()):
/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
1088 recording_scopes = False
1089 try:
-> 1090 result = self.forward(*input, **kwargs)
1091 finally:
1092 if recording_scopes:
/usr/local/lib/python3.8/dist-packages/snntorch/_neurons/leaky.py in forward(self, input_, mem)
159 if self.init_hidden:
160 self._leaky_forward_cases(mem)
--> 161 self.reset = self.mem_reset(self.mem)
162 self.mem = self.state_fn(input_)
163
/usr/local/lib/python3.8/dist-packages/snntorch/_neurons/neurons.py in mem_reset(self, mem)
86 """Generates detached reset signal if mem > threshold.
87 Returns reset."""
---> 88 mem_shift = mem - self.threshold
89 reset = self.spike_grad(mem_shift).clone().detach()
90
/usr/local/lib/python3.8/dist-packages/poptorch/_poplar_executor.py in __torch_function__(cls, func, types, args, kwargs)
279 if kwargs is None:
280 kwargs = {}
--> 281 return super().__torch_function__(func, types, args,
282 kwargs)
283
RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient
Tensor:
(1,1,.,.) =
Columns 1 to 9 0.4529 0.4529 0.4529 0.4529 0.4529 0.4529 0.4529 0.4529 0.4529 ....
With self.lif = snn.Leaky(beta=0.95, spike_grad=surrogate.fast_sigmoid(), init_hidden=True, learn_beta=True, learn_threshold=True)
Add new tests for BNTT1d and 2d.
Hello!
I am trying out tutorial_6 (https://colab.research.google.com/github/jeshraghian/snntorch/blob/master/examples/tutorial_6_CNN.ipynb) but the only difference is I am using CIFAR-10 dataset instead of MNIST. I trained it for 10 epochs but the accuracy wasn't good, so I tried training for 50 epochs and the accuracy is still around 35-40 %. Can anyone please guide me on resolving this issue ?
Hi Jason,
First of all, I appreciate your wonderful effort in developing this package and a detailed documentation. I have recently started using snntorch for rate based SNN coding. Although I am getting good performance for purely software based run, I am facing issues with inputing custom weights extracted from a synaptic device. My accuracy is getting stuck at around 10% which is the same as the untrained accuracy.
I used a custom function to input the weights from a text file as shown in the screenshot. Please let me know how to solve this issue.
NB: I am pretty new to programming. so pls excuse me if my code is too cumbersome :)
`
Here is the full file and the text file for data input
rate_SNN_dev_weights.zip
Thanks,
Kannan
spikegen.rate and spikegen.rate_conv do not return equal dimensions.
One unsqueezes the output in the first dim.
spk_in = spikegen.rate_conv(torch.rand((200, 784)))
The tutorial_1 code in Colab looks like based on an out-of-date snntorch version.
For example, the code of rate encoding in Colab is
spike_data, spike_targets = spikegen.rate(data_it, targets_it, num_outputs=num_outputs, num_steps=num_steps,
gain=1, offset=0, one_hot=False, time_varying_targets=False)
While the up-to-date code in Github is
spike_data = spikegen.rate(data_it, num_steps=num_steps, gain=1, offset=0).
As I tried, both from this link https://snntorch.readthedocs.io/en/latest/examples.html and clicking the open in colab icon will lead to the wrong version tutorials.
A quick solution is directly cloning the repository and running locally or loading the notebook to Colab via Github link according to https://colab.research.google.com/github/googlecolab/colabtools/blob/master/notebooks/colab-github-demo.ipynb#:~:text=Colab%20can%20load%20public%20github,with%20no%20required%20authorization%20step.
the inhibition
argument of neurons is only setup for single dimensional channels / outputs of nn.Linear
It does not work for convolutional layers.
The way I see it, there are two options:
I tried to use the surrogate gradient function by running the following line of code:
spike_grad = snn.FastSigmoid.apply
However, I am getting an error stating that "module 'snntorch' has no attribute 'FastSigmoid'".
I see that it is not implemented yet to the neuron models in your init.py file, may I ask when the push is going to be done?
Moreover, I see that we have only examples of applying rate encoding to the MNIST dataset, I need to ask if it is possible to have an example showing how to use the Latency encoding may also be on a Convolution network?
Thanks in advance.
I would like to install snntorch. I found it requires poptorch, which is dependent on Popular SDK.
How can I download Popular SDK? GraphCore offers Popular SDK only for signed-up users.
For this reason, I cannot download Popular SDK... Please help me.
I am developing an SNN model to substitute ANNs with SNNs. I have observed that the performance of SNNs cannot catch up that of ANNs. To make SNNs comparable to ANNs, I tried the following:
I found that these a bit contribute to the performance of SNNs.
Do you have any other tips to convert ANNs to SNNs when it comes to setting hyperparameters?
Enable user to pass in state equations, to specify the state variable(s), and custom neuron class is automatically generated which can be treated like any other LIF neuron.
Hi Jason,
I didn't find the program code to encode the input data in tutorial 5, nor in snn.Leaky. Does this mean that the user can input data into the model in Tutorial 5 without coding the input data into spike train?
The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.
possible solutions:
SConv2dLSTM currently concatenates the hidden cell with the input cell, applies a single nn.Conv2d function of channel length 4*out_channel
The output is then split into 4 chunks (along the channel dimension), which are fed to each of the 4 LSTM gates
This means HxW of the hidden cell must be equal to HxW of input
Stride != 1 causes HxW to no longer be equal.
To fix this, a stride argument can trigger the conv to take place before concatenation of the hidden state mem
and input x
.
Far less efficient than running all conv operations in one hit, though I don't yet see a way around this.
Good day!
I wanted to know what does mean num_steps = 300. Does it mean that to use only 300 events?
Like we have 60000 samples in NMNIST, and each sample has around 4500 events. But if we set dt =1000,
and num_steps = 300, so we use only 300 events out of 4500?
Thank you!
What parameters are provided to compare the difference in energy consumption between the two, such as the amount of computation?
Hello,
I am trying to use the RSynaptic neuron model but even though it is documented in the snnTorch website, the class RSynaptic(LIF)
is not included in the snntroch
module.
This this the error I get when I try to use it:
AttributeError: module 'snntorch' has no attribute 'RSynaptic'
No unit tests for the Lapicque neuron model.
No errors thrown - just missing feature.
Is snntorch loss suitable for regression loss function? Because I found that snn loss seems only suitable for classification loss.
Add an adaptive, time-varying threshold to all neuron models.
threshold should decay back to steady-state value over some time-constant
threshold may need to saturate to some value to avoid removing firing activity
run tests to determine if it should be detached from the computational graph (likely to be the case, as it will only be activated upon spike firing)
Getting around this should be quite straightforward:
num_steps
into an optional argument (but an error is raised if input data is time-series)num_steps
, instead use data.size(0)
Originally posted by msbouanane November 26, 2021
Hello,
The backprop.BPTT function requires the num_steps parameter. But when using a neuromorphic dataset, the num_steps will be variable depending on the longest recording in the mini-batch.
What should I pass to this parameter instead?
When I try to reproduce one of the example, e.g.
import snntorch as snn
a = torch.Tensor([1, 2, 2.9, 3, 3.9])
snn.spikegen.delta(a, threshold=1)
I get the error module 'snntorch' has no attribute 'spikegen'
. Where do I find the module?
when i try to use TBPTT or RTRL from backprop on time varying signal, it gives me this error:
if K_flag is False:
UnboundLocalError: local variable 'K_flag' referenced before assignment
In the latest updates, I see that the num_output
has been removed from the spikegen
. No clue why!
Moreover, also when I just decide about removing the num_output
parameter. I get another error from the num_steps
when I set it to any int value?
BTW, the same exists if I tried to do rate encoding instead of latency encoding.
Here: you will find a quick ipynb file that shows the errors.
automate dataset download/extraction/transform process in module spikevision
.
While I am looking through your tutorials, I have noticed that in tutorial 6, the following line:
Part of this tutorial was inspired by Friedemann Zenke’s extensive work on SNNs. Check out his repo on surrogate gradients here, and a favourite paper...
seems to have an unresolved URL https://github.corm/fzenke/spytorch
, which I think 'corm' is misspelled.
Click the link.
What is the power efficiency between the two, the amount of computation or some other parameter
It is possible to have neurons firing continuously using the default reset mechanism or if using RLeaky and reset to zero. The latter is due to only resetting the input but not the recurrent connections. This is undesirable behavior as it allows the neurons to essentially not be spiking neurons given the right weight values.
I think that ffmpeg
is missing from the dependencies listed in README.rst
.
Using conda, I installed the dependencies and then received the following error on the video conversion step of the first tutorial:
RuntimeError: Requested MovieWriter (ffmpeg) not available
This was easily sorted with: conda install -c conda-forge ffmpeg
.
The citation included in the README file has an incorrect reference:
@article{eshraghian2021training,
title={Training spiking neural networks using lessons from deep learning},
author={Eshraghian, Jason K and Ward, Max and Neftci, Emre and Wang, Xinxin
and Lenz, Gregor and Dwivedi, Girish and Bennamoun, Mohammed and Jeong, Doo Seok
and Lu, Wei D},
journal={arXiv preprint arXiv:1906.09395},
year={2021}
}
I assume the correct arXiv code should be: 2109.12894:
@article{eshraghian2021training,
title={Training spiking neural networks using lessons from deep learning},
author={Eshraghian, Jason K and Ward, Max and Neftci, Emre and Wang, Xinxin
and Lenz, Gregor and Dwivedi, Girish and Bennamoun, Mohammed and Jeong, Doo Seok
and Lu, Wei D},
journal={arXiv preprint arXiv:2109.12894},
year={2021}
}
Other than fucntional.mse_membrane_loss()
, all other loss functions available are only able to train rate-coded networks.
Temporal / time-to-first-spike mechanisms are more challenging as they involve a conversion from the spike (1/0) domain to the time domain. The most obvious fix is to simply iterate through the output spike tensor's time-axis, and populate a new tensor with the time at which each spike occurs. This would then be passed into a MSELoss function.
But indexing breaks the computational graph in PyTorch, so this would have to be overridden using a pre-defined backward
function that is inherited from torch.autograd.Function
.
Bonus features:
While implementing the LeakySigmoidSurrogate model presented in tutorial 6 I came across different outputs from my personal implementation following the Colab Notebook and snn.Leaky() with the same configurations.
After diving into snn.Leaky() implementation I realised that the Net forward function is not the same.
Current forward
spk = self.surrogate_func((mem - self.threshold), self.slope) # call the Heaviside function
reset = (spk - self.threshold).clone().detach()
mem = self.beta * mem + input_ - reset
snn.Leaky() forward
mem_shift = mem - self.threshold
reset = self.surrogate_func(mem_shift, self.slope).clone().detach()
mem = self.beta * (mem - reset * self.threshold) + input_
spk = self.surrogate_func(mem_shift, self.slope) # call the Heaviside function
As well I think that in the init instead of super(Leaky_Surrogate, self).__init__()
should be super(LeakySigmoidSurrogate, self).__init__()
right now, spikegen.rate
, spikegen.latency
and spikegen.delta
accept targets as an input.
But there is no way to specify how to encode the targets beyond one-hot-encodings & time-varying data.
spikegen.rate
or spikegen.latency
.Hi,
I 'm doing some latency code training with snntorch. I'm not very sure how to calculate the loss of temporal coding. In rate coding, we just sum the loss of ervery time steps, but how to calculate the loss in temporal coding?
Thanks.
Enable user to pass in an arbitrary function which is automatically centered about the firing threshold.
Every 0.5s: nvidia-smi neuro: Fri Dec 2 11:16:53 2022
Fri Dec 2 11:16:53 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.82.01 Driver Version: 470.82.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA RTX A6000 On | 00000000:1B:00.0 Off | Off |
| 30% 30C P8 29W / 300W | 1MiB / 48682MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA RTX A6000 On | 00000000:1C:00.0 Off | Off |
| 30% 27C P8 22W / 300W | 1MiB / 48685MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 NVIDIA RTX A6000 On | 00000000:1D:00.0 Off | Off |
| 30% 32C P8 23W / 300W | 1MiB / 48685MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 3 NVIDIA RTX A6000 On | 00000000:1E:00.0 Off | Off |
| 30% 31C P8 23W / 300W | 1MiB / 48685MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 4 NVIDIA RTX A6000 On | 00000000:3D:00.0 Off | Off |
| 30% 27C P8 22W / 300W | 1MiB / 48685MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 5 NVIDIA RTX A6000 On | 00000000:3F:00.0 Off | Off |
| 30% 29C P8 23W / 300W | 1MiB / 48685MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 6 NVIDIA RTX A6000 On | 00000000:40:00.0 Off | Off |
| 30% 27C P8 22W / 300W | 1MiB / 48685MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 7 NVIDIA RTX A6000 On | 00000000:41:00.0 Off | Off |
| 30% 30C P8 22W / 300W | 1MiB / 48685MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
I'm trying to train NMNIST with snntorch using multi GPU. since snntorch is based on torch package, I thought data parrallel from torch nn should work.
here's whole code.
import torch
import torchvision.datasets as dsets
import torchvision.transforms as transforms
import torch.nn.init
import os
import torch.nn as nn
import time
import matplotlib.pyplot as plt
import tonic.transforms as transforms
import tonic
import numpy as np
import snntorch as snn
from snntorch import surrogate
from snntorch import functional as SF
from snntorch import spikeplot as splt
from snntorch import utils
import torch.nn as nn
import os
from torch.utils.data import DataLoader, random_split
import torch
device = 'cuda' if torch.cuda.is_available() else 'cpu'
sensor_size = tonic.datasets.NMNIST.sensor_size
# Denoise removes isolated, one-off events
# time_window
frame_transform = transforms.ToFrame(sensor_size=sensor_size, time_window=1)
frame_transform = transforms.Compose([transforms.Denoise(filter_time=10000),
transforms.ToFrame(sensor_size=sensor_size,
time_window=50000)
])
trainset = tonic.datasets.NMNIST(save_to='/home/hubo1024/PycharmProjects/snntorch/data/NMNIST', transform=frame_transform, train=True)
testset = tonic.datasets.NMNIST(save_to='./home/hubo1024/PycharmProjects/snntorch/data/NMNIST', transform=frame_transform, train=False)
# seed fix
torch.manual_seed(777)
# seed fix if gpu is available
if device == 'cuda':
torch.cuda.manual_seed_all(777)
#batch_size = 100
batch_size = 32
dataset_size = len(trainset)
train_size = int(dataset_size * 0.9)
validation_size = int(dataset_size * 0.1)
trainset, valset = random_split(trainset, [train_size, validation_size])
print(len(valset))
print(len(trainset))
trainloader = DataLoader(trainset, batch_size=batch_size, collate_fn=tonic.collation.PadTensors(), shuffle=True)
valloader = DataLoader(valset, batch_size=batch_size, collate_fn=tonic.collation.PadTensors(), shuffle=True)
testloader = DataLoader(testset, batch_size=batch_size, collate_fn=tonic.collation.PadTensors())
spike_grad = surrogate.fast_sigmoid(slope=75)
beta = 0.5
class CNN(torch.nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.keep_prob = 0.5
self.layer1 = torch.nn.Sequential(
nn.Conv2d(2, 12, 5),
nn.MaxPool2d(2),
snn.Leaky(beta=beta, spike_grad=spike_grad, init_hidden=True)
)
self.layer2 = torch.nn.Sequential(
nn.Conv2d(12, 32, 5),
nn.MaxPool2d(2),
snn.Leaky(beta=beta, spike_grad=spike_grad, init_hidden=True)
)
self.layer4 = torch.nn.Sequential(
nn.Flatten(),
nn.Linear(32 * 5 * 5, 10),
snn.Leaky(beta=beta, spike_grad=spike_grad, init_hidden=True, output=True)
)
def forward(self, data):
spk_rec = []
layer1_rec = []
layer2_rec = []
utils.reset(self.layer1) # resets hidden states for all LIF neurons in net
utils.reset(self.layer2)
utils.reset(self.layer4)
for step in range(data.size(1)): # data.size(0) = number of time steps
input_torch = data[:, step, :, :, :]
input_torch = input_torch.cuda()
#print(input_torch)
out = self.layer1(input_torch)
#out1 = out
out = self.layer2(out)
#out2 = out
out, mem = self.layer4(out)
#out = self.layer4(out)
spk_rec.append(out)
#layer1_rec.append(out1)
#layer2_rec.append(out2)
return torch.stack(spk_rec)#, torch.stack(layer1_rec), torch.stack(layer2_rec)
model = CNN().to(device)
device_ids = [0, 1] #your GPU index
model = torch.nn.DataParallel(model, device_ids=device_ids)
#model = nn.DataParallel(model).to(device)
optimizer = torch.optim.NAdam(model.parameters(), lr=0.005,betas=(0.9, 0.999))
loss_fn = SF.mse_count_loss(correct_rate=0.8, incorrect_rate=0.2)
#model = nn.DataParallel(model)
total_batch = len(trainloader)
print('총 배치의 수 : {}'.format(total_batch))
loss_fn = SF.mse_count_loss(correct_rate=0.8, incorrect_rate=0.2)
num_epochs = 15
loss_hist = []
acc_hist = []
v_acc_hist = []
t_spk_rec_sum = []
start = time.time()
val_cnt = 0
v_acc_sum= 0
avg_loss = 0
index = 0
#################################################
for epoch in range(num_epochs):
torch.save(model.state_dict(), '/home/hubo1024/PycharmProjects/snntorch/model_pt/Radam_15epoch-50000.pt')
for i, (data, targets) in enumerate(iter(trainloader)):
data = data.cuda()
targets = targets.cuda()
model.train()
spk_rec = model(data)
#print(spk_rec.shape)
loss_val = loss_fn(spk_rec, targets)
avg_loss += loss_val.item()
optimizer.zero_grad()
loss_val.backward()
optimizer.step()
# Store loss history for future plotting
loss_hist.append(loss_val.item())
val_cnt = val_cnt+1
#del loss_val
if val_cnt == len(trainloader)/2-1:
val_cnt=0
for ii, (v_data, v_targets) in enumerate(iter(valloader)):
v_data = v_data.to(device)
v_targets = v_targets.to(device)
v_spk_rec = model(v_data)
#
# print(t_spk_rec.shape)
v_acc = SF.accuracy_rate(v_spk_rec, v_targets)
del v_spk_rec
if ii == 0:
v_acc_sum = v_acc
cnt = 1
else:
v_acc_sum += v_acc
cnt += 1
#del v_acc
plt.plot(acc_hist)
plt.plot(v_acc_hist)
plt.legend(['train accuracy', 'validation accuracy'])
plt.title("Train, Validation Accuracy-Radam 15epoch-50000")
plt.xlabel("Iteration")
plt.ylabel("Accuracy")
# plt.show()
plt.savefig('Radam_15epoch-50000.png')
plt.clf()
v_acc_sum = v_acc_sum/cnt
# avg_loss = avg_loss / (len(trainloader) / 2)
# print('average loss while half epoch', avg_loss)
# if avg_loss <= 0.5:
# index = 1
# break
# else:
# avg_loss = 0
# index = 0
print('Radam-15epoch-50000')
print("time :", time.time() - start,"sec")
print(f"Epoch {epoch}, Iteration {i} \nTrain Loss: {loss_val.item():.2f}")
acc = SF.accuracy_rate(spk_rec, targets)
acc_hist.append(acc)
v_acc_hist.append(v_acc_sum)
print(f"Train Accuracy: {acc * 100:.2f}%")
print(f"Validation Accuracy: {v_acc_sum * 100:.2f}%\n")
# if index == 1:
# break
# if index == 1:
# break
# 학습을 진행하지 않을 것이므로 torch.no_grad()
'''
with torch.no_grad():
X_test = mnist_test.test_data.view(len(mnist_test), 1, 28, 28).float().to(device)
Y_test = mnist_test.test_labels.to(device)
prediction = model(X_test)
correct_prediction = torch.argmax(prediction, 1) == Y_test
accuracy = correct_prediction.float().mean()
print('Accuracy:', accuracy.item())
'''
and here's error
(snn_torch) hubo1024@neuro:~/PycharmProjects/snntorch$ CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0,1,2,3,4,5 python gpu_6_run.py
6000
54000
총 배치의 수 : 13500
Traceback (most recent call last):
File "/home/hubo1024/PycharmProjects/snntorch/gpu_6_run.py", line 146, in <module>
spk_rec = model(data)
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 168, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 178, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
output.reraise()
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/_utils.py", line 461, in reraise
raise exception
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
output = module(*input, **kwargs)
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hubo1024/PycharmProjects/snntorch/gpu_6_run.py", line 102, in forward
out = self.layerconv1(input_torch)
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/nn/modules/container.py", line 139, in forward
input = module(input)
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/snntorch/_neurons/leaky.py", line 162, in forward
self.mem = self.state_fn(input_)
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/snntorch/_neurons/leaky.py", line 201, in _build_state_function_hidden
self._base_state_function_hidden(input_) - self.reset * self.threshold
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/snntorch/_neurons/leaky.py", line 195, in _base_state_function_hidden
base_fn = self.beta.clamp(0, 1) * self.mem + input_
File "/home/hubo1024/anaconda3/envs/snn_torch/lib/python3.9/site-packages/torch/_tensor.py", line 1121, in __torch_function__
ret = func(*args, **kwargs)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
I rerun this code after removing snn.Leaky layer in CNN and it worked fine. (of course the cost doesn't converge and accuracy was 0% but still it runs)
So I assume that the reason of this error is snn.Leaky layer.
I think changing
I was wondering if anyone had used snnTorch for regression, and perhaps how you set your networks up. Just looking for simple, general examples! MSELoss would likely be the type of loss used as I see it.
Describe what you were trying to get done.
Tell us what happened, what went wrong, and what you expected to happen.
Paste the command(s) you ran and the output.
If there was a crash, please include the traceback here.
Apply on/off target weighting to snntorch.functional
losses in the same way the PyTorch enables weighting.
Cross Entropy-based losses should be straightforward; Mean Square Error Losses are less trivial.
This seems to be a super popular feature request.
Making accurate estimates seems near impossible, but we can probably generate an order of magnitude guess here.
The user would construct a model, pass data in, and the power profiling function returns the number of Synaptic operations in the forward-pass (this could be averaged across batches).
Each synaptic op would be scaled by the energy cost for all selected devices; e.g., various GPUs & neuromorphic hardware.
The same number would be given for non-spiking networks too. This could be achieved by just removing the spiking modules.
SpikingKeras has a similar function that does it really nicely. However, it overstates the improvement given with spikes because it does not account for overhead (i.e., moving data to/from memory, or between multiple chips).
Including an argument that factors in overhead would by tricky, but useful.
The model would be parsed for number of neurons/synapses, and if either exceeds the bandwidth of a single chip, then we need to estimate how frequently data needs to be moved between chips & add that to the overall energy consumption.
A lot of coarse estimates would be made, but I think it could be helpful.
Hi,
I tried to use the snnTorch to do exactly as you are doing in Tutorial 3 (without spike_grad), and also in the upcoming Tutorial 4 (applied spike_grad to stein neuron) for Spiking CNNs. Moreover, I also converted my dataset from static to spike version as in Tutorial 4 using rate encoding.
However, usually, I found that the network learns nothing and the training loss goes down and up slightly, but the output numbers and accuracy are completely unintuitive.
For example, I used a Conventional CNN on my Dataset and I got 85% accuracy, and I used it with the same architecture and hyperparameters and nothing is learned.
I will be glad if you can have some direct suggestions about what may be the problem and why the network is learning nothing in the Spiking domain?
Looking forward to your response
At "Tutorial 7 - NEUROMORPHIC DATASETS WITH TONIC + SNNTORCH", shape of event_tensor, made by torch.utils.data.DataLoader, is depicted as torch.Size([311, 128, 2, 34, 34]). But when I run the codes, shape of event_tensor shown as torch.Size([128, 311, 2, 34, 34]), first and second dimensions are changed. So I couldn't train anything, because tensor shape is not matched exactly.
Why does the shape of tutorial look different from the actual implementation?
Was there any update of snnTorch or torch(DataLoader)?
Here is colab error message. I didn't modify anything, just ran tutorial code.
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/loss.py:530: UserWarning: Using a target size (torch.Size([128, 10])) that is different to the input size (torch.Size([312, 10])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
return F.mse_loss(input, target, reduction=self.reduction)
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
[<ipython-input-18-5a0ce5857296>](https://localhost:8080/#) in <module>
13 net.train()
14 spk_rec = forward_pass(net, data)
---> 15 loss_val = loss_fn(spk_rec, targets)
16
17 # Gradient calculation + weight update
4 frames
[/usr/local/lib/python3.7/dist-packages/torch/functional.py](https://localhost:8080/#) in broadcast_tensors(*tensors)
71 if has_torch_function(tensors):
72 return handle_torch_function(broadcast_tensors, tensors, *tensors)
---> 73 return _VF.broadcast_tensors(tensors) # type: ignore[attr-defined]
74
75
RuntimeError: The size of tensor a (312) must match the size of tensor b (128) at non-singleton dimension 0
Issue: The present implementation of RLeaky
and RSynaptic
each assign only a single weight per neuron in the recurrent feedback connection.
Proposed Fix: This should be modified to account for cases where the linear connection should be fed through a parameterizable dense (and possibly conv?) recurrent connection.
Possible implementation:
lif1 = snn.RLeaky(beta=beta, dense=128)
# each output spike is looped back to the input via a dense layer of 128 neurons
Challenge with above implementation:
This cannot be applied to convolutional layers. The shape of the output tensor must be projected back to whatever the input tensor is such that the two can be added or appended. (Q: which one?)
For convs, perhaps the input data has a 'size check' to infer how padding, stride, kernel size must be applied.
Ideally, the user could specify kernel size and everything else is inferred (e.g., depth).
We already do something similar for the spiking Conv2d LSTM neuron, too.
Hello Jason,
When I ran the code in Tutorial 2, I found that several ploting functions were undefined. Can you tell me which package they are included in so that I can install the missing package?
Looking forward to your reply.
Thanks,
Sasuke
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.