ooooverflow / bisenet Goto Github PK
View Code? Open in Web Editor NEWBiSeNet based on pytorch
BiSeNet based on pytorch
can you please give an sample of class_dict.csv ?
class DiceLoss(nn.Module): def __init__(self): super().__init__() self.epsilon = 1e-5
def forward(self, output, target):
# print(output.shape)
# print(target.shape)
assert output.size() == target.size(), "'input' and 'target' must have the same shape"
# 在classes上做softmax
output = F.softmax(output, dim=1)
# 打平tensor
output = flatten(output) # [num_classes,B*H*W]
target = flatten(target) # [num_classes,B*H*W]
# intersect = (output * target).sum(-1).sum() + self.epsilon
# denominator = ((output + target).sum(-1)).sum() + self.epsilon
intersect = (output * target).sum(-1)
denominator = (output + target).sum(-1)
# dice --(0-0.5)
dice = intersect / denominator
dice = torch.mean(dice)
# 1-dice (0.5,1)???
return 1 - dice
# return 1 - 2. * intersect / denominator
double the intersection over union?
i train with context_path: resnet101 and resnet 18.
first question: both the validation accuracy hardly reach 0.9, Mostly stop as 0.88-0.89.
second question: while the resnet101 training, the validation accuracy will fluctuate a lot after epoch 100, and will drop a lot after about epoch 180+.
could you please share your training parameter like lr,batchsize,GPU num,crop_height,crop_width and some detailed trick?
BiSeNet$ python demo.py
Traceback (most recent call last):
File "demo.py", line 81, in
main(params)
File "demo.py", line 60, in main
model.module.load_state_dict(torch.load(args.checkpoint_path))
File "/home/keshi/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 381, in load
f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/ckpt/'
(base) keshi@keshi-Blade:~/BiSeNet$ python train.py
Traceback (most recent call last):
File "train.py", line 209, in
main(params)
File "train.py", line 148, in main
loss=args.loss, mode='train')
File "/home/keshi/BiSeNet/dataset/CamVid.py", line 43, in init
self.label_info = get_label_info(csv_path)
File "/home/keshi/BiSeNet/utils.py", line 31, in get_label_info
ann = pd.read_csv(csv_path)
File "/home/keshi/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 685, in parser_f
return _read(filepath_or_buffer, kwds)
File "/home/keshi/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 457, in _read
parser = TextFileReader(fp_or_buf, **kwds)
File "/home/keshi/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 895, in init
self._make_engine(self.engine)
File "/home/keshi/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1135, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "/home/keshi/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1917, in init
self._reader = parsers.TextReader(src, **kwds)
File "pandas/_libs/parsers.pyx", line 382, in pandas._libs.parsers.TextReader.cinit
File "pandas/_libs/parsers.pyx", line 689, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: [Errno 2] File b'/path/to/CamVid/class_dict.csv' does not exist: b'/path/to/CamVid/class_dict.csv'
Could you tell me why this error will happen? How could I solve this problem?
欢迎加入深度学习双目视觉,群聊号码:1018698420
In 3.1 Spatial Path, there is a sentence "Based on this observation, we propose a Spatial Path to preserve the spatial size of the original input image and encode affluent spatial information. ". But in 3.1 Spatial Path, this path extracts the output feature maps that is 1/8 of the original image. What does it mean to preserve the spatial size of the original input image? Why does the paper say to preserve the spatial size of the original input image?
I'm looking forward to your reply. I'm very confused about this question. Thank you very much!
train.py里面并没有实现epoch_start_i函数判断的模块,所以当引用这个参数会报错。
因此目前来说,如果想接着上次迭代的epoch来进行训练的话,只能通过pre-train model 来操作
The loss function in the original paper is composed of two parts. But you only use the output of feature fusion model to calculate the loss. And the loss they use is cross entropy, here you use is binary cross entropy. Is there any reason to these changes? Thanks!
then you can ignore pycharm folder and others. Pycharm makes python more convenient.
Hi, I convert label from RGB mode to 'P' mode based on class_dict.csv
, and found there are some points/coord with wrong label, aka its color is out of color defined in class_dict.csv
. Here is how I convert RGB mode to 'P' mode and check label:
from PIL import Image
from utils import get_label_info
labelinfo = get_label_info(csvfile)
palette = []
for key in labelinfo:
for item in labelinfo[key]:
palette.append(item)
ref_image = Image.new(mode='P', size=(1, 1))
ref_image.putpalette(palette)
img_rgb = Image.open('rgb_mode.png')
img_p = img_rgb.quantize(palette=ref_image)
img_p_wrong = img_p[img > 31]
img_rgb_wrong = img_rgb[img > 31]
I'm not sure how to deal with this image, currently I simply remove it out of my training/val/testing. After convert RGB mode to 'P' mode, I don't need to use one hot encoding for label anymore. It seems that the one hot encoding & decoding slow down training speed.
And, the code you calculating accuracy seems weird, https://github.com/ooooverflow/BiSeNet/blob/master/utils.py#L103, pred
& label
are in shape of 1xCxHxW
, where C is the channels, which is 3 in this case. pred[:, :, h, w] == label[:, :, h, w]
means one pixel prediction right, not three.
Hello, I found that you had deleted the pretrained model, so if I want to run demo can only train by myself? can you publish your pretrained model please? Thank you very much!
when i train my data , it happend as followed:
os@os-l3:/disk3t-2/zym/BiSeNet-PyTorch$ python train.py
epoch 0, lr 0.001000: 0%| | 0/4963 [00:00<?, ?it/s]Traceback (most recent call last):
File "train.py", line 157, in
main(params)
File "train.py", line 141, in main
train(args, model, optimizer, dataloader_train, dataloader_val, csv_path)
File "train.py", line 56, in train
output = model(data)
File "/home/os/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/os/.local/lib/python2.7/site-packages/torch/nn/parallel/data_parallel.py", line 121, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/os/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/disk3t-2/zym/BiSeNet-PyTorch/model/build_BiSeNet.py", line 97, in forward
cx1 = self.attention_refinement_module1(cx1)
File "/home/os/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/disk3t-2/zym/BiSeNet-PyTorch/model/build_BiSeNet.py", line 40, in forward
assert self.in_channels == x.size(1), 'in_channels and out_channels should all be {}'.format(x.size(1))
AssertionError: in_channels and out_channels should all be 256
Check out here: https://github.com/ycszen/TorchSeg
load model from ./checkpoints/epoch_295.pth ...
Traceback (most recent call last):
File "c:\Users\Administrator\Desktop\BiSeNet-master\BiSeNet-master\demo.py", line 80, in
main(params)
File "c:\Users\Administrator\Desktop\BiSeNet-master\BiSeNet-master\demo.py", line 60, in main
model.module.load_state_dict(torch.load(args.checkpoint_path))
File "C:\Program Files\Python\lib\site-packages\torch\nn\modules\module.py", line 518, in getattr
type(self).name, name))
AttributeError: 'BiSeNet' object has no attribute 'module'
PS C:\Users\Administrator\Desktop\BiSeNet-master\BiSeNet-master>
Where do I put the dataset ?
Do you have any pretrained weights for COCO dataset. If not do you have any train.py or script to load the COCO dataset. Appreciate any help.
May I know how to test speed (fps)?thank you all!
When I first trained, the training data(MIOU) was normal.
but second, miou has been fixed at around 0.12, I thought for a long time but did't solve this problem.
So, I want to ask if you have encountered this problem.
AttributeError: 'tuple' object has no attribute 'log_softmax'
The original weight has expired, So, Do you have any plans for a new release recently?
when i use the model epoch_295.pth in GoogleDrive and put it in ./checkpoints ,run eval.py, it comes a error like this:
load model from /root/data/Seg_Pytorch/BiSeNet-master/checkpoints/epoch_295.pth ...
Traceback (most recent call last):
File "eval.py", line 97, in
main(params)
File "eval.py", line 82, in main
model.module.load_state_dict(torch.load(args.checkpoint_path))
File "/root/miniconda3/envs/pytorch10/lib/python3.6/site-packages/torch/nn/modules/module.py", line 721, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for BiSeNet:
Missing key(s) in state_dict: "supervision1.weight", "supervision1.bias", "supervision2.weight", "supervision2.bias".
Unexpected key(s) in state_dict: "saptial_path.convblock1.bn.num_batches_tracked", "saptial_path.convblock2.bn.num_batches_tracked", "saptial_path.convblock3.bn.num_batches_tracked", "context_path.features.bn1.num_batches_tracked", "context_path.features.layer1.0.bn1.num_batches_tracked", "context_path.features.layer1.0.bn2.num_batches_tracked", "context_path.features.layer1.0.bn3.num_batches_tracked", "context_path.features.layer1.0.downsample.1.num_batches_tracked",
You said the net structure has been modified, so the pretrained model does not match the current model, so do you have any other pre-trained mode recommend for this network?
pred and label both are 3 channel,so why do you directly flatten() them?
Hello, I can't find this file.
Firstly,you should give the version of pytorch and python,which is enjoyable;
Secondly,what's the speed of your model?Is it near to105FPS?
Traceback (most recent call last):
File "train.py", line 209, in
main(params)
File "train.py", line 148, in main
loss=args.loss, mode='train')
File "/home/lr/代码/BiSeNet-master/dataset/CamVid.py", line 43, in init
self.label_info = get_label_info(csv_path)
File "/home/lr/代码/BiSeNet-master/utils.py", line 31, in get_label_info
ann = pd.read_csv(csv_path)
File "/home/lr/anaconda3/envs/gpu/lib/python3.7/site-packages/pandas/io/parsers.py", line 685, in parser_f
return _read(filepath_or_buffer, kwds)
File "/home/lr/anaconda3/envs/gpu/lib/python3.7/site-packages/pandas/io/parsers.py", line 457, in _read
parser = TextFileReader(fp_or_buf, **kwds)
File "/home/lr/anaconda3/envs/gpu/lib/python3.7/site-packages/pandas/io/parsers.py", line 895, in init
self._make_engine(self.engine)
File "/home/lr/anaconda3/envs/gpu/lib/python3.7/site-packages/pandas/io/parsers.py", line 1135, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "/home/lr/anaconda3/envs/gpu/lib/python3.7/site-packages/pandas/io/parsers.py", line 1906, in init
self._reader = parsers.TextReader(src, **kwds)
File "pandas/_libs/parsers.pyx", line 380, in pandas._libs.parsers.TextReader.cinit
File "pandas/_libs/parsers.pyx", line 687, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: [Errno 2] File b'/path/to/CamVid/class_dict.csv' does not exist: b'/path/to/CamVid/class_dict.csv'
Hi, I have a question,Is this error due to lack of dataset?
RuntimeError: Error(s) in loading state_dict for BiSeNet:
Missing key(s) in state_dict: "supervision1.weight", "supervision1.bias", "supervision2.weight", "supervision2.bias".
1.Have you trained your model on CItyscape dataset? When I change the dataset from Camvid to Cityscape, run the train.py, i get an error like this:
epoch 0, lr 0.001000: 0%| | 0/744 [00:00<?, ?it/s]Traceback (most recent call last): File "train.py", line 206, in <module> main(params) File "train.py", line 187, in main train(args, model, optimizer, dataloader_train, dataloader_val, csv_path) File "train.py", line 86, in train for i,(data, label) in enumerate(dataloader_train): File "/home/sunjingchen/miniconda3/envs/pytorch041/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 336, in __next__ return self._process_next_batch(batch) File "/home/sunjingchen/miniconda3/envs/pytorch041/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 357, in _process_next_batch raise batch.exc_type(batch.exc_msg) ValueError: Traceback (most recent call last): File "/home/sunjingchen/miniconda3/envs/pytorch041/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 106, in _worker_loop samples = collate_fn([dataset[i] for i in batch_indices]) File "/home/sunjingchen/miniconda3/envs/pytorch041/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 106, in <listcomp> samples = collate_fn([dataset[i] for i in batch_indices]) File "/home/sunjingchen/BiSeNet-City/dataset/CamVid.py", line 81, in __getitem__ label = one_hot_it(label, self.label_info).astype(np.uint8) File "/home/sunjingchen/BiSeNet-City/utils.py", line 44, in one_hot_it equality = np.equal(label, color) ValueError: operands could not be broadcast together with shapes (640,640,4) (3,)
Could you tell me what's the reason of this error? I think maybe is the format of Camvid dataset is different from Cityscape dataset.
2.Have you tested the speed of BiSeNet on Camvid, Is it near to105FPS?
Thank you very much!
I am trying to run this code, but I get the following error:
cx = torch.cat((cx1, cx2), dim=1)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 128 and 64 in dimension 2 at /pytorch/aten/src/THC/generic/THCTensorMath.cu:71
Any idea, why is it so?
FileNotFoundError Traceback (most recent call last)
in
5 #df
6 for site_name in df['SiteName'].unique():
----> 7 df[df['SiteName'] == site_name].to_csv('{}.csv'.format(site_name))
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\generic.py in to_csv(self, path_or_buf, sep, na_rep, float_format, columns, header, index, index_label, mode, encoding, compression, quoting, quotechar, line_terminator, chunksize, tupleize_cols, date_format, doublequote, escapechar, decimal)
3018 doublequote=doublequote,
3019 escapechar=escapechar, decimal=decimal)
-> 3020 formatter.save()
3021
3022 if path_or_buf is None:
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\io\formats\csvs.py in save(self)
155 f, handles = _get_handle(self.path_or_buf, self.mode,
156 encoding=self.encoding,
--> 157 compression=self.compression)
158 close = True
159
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\io\common.py in _get_handle(path_or_buf, mode, encoding, compression, memory_map, is_text)
422 elif encoding:
423 # Python 3 and encoding
--> 424 f = open(path_or_buf, mode, encoding=encoding, newline="")
425 elif is_text:
426 # Python 3 and no explicit encoding
FileNotFoundError: [Errno 2] No such file or directory: '/PI_Blackfriars_Sys_1_4/Room_34_SetTempHeat.csv'
can you please tell me why this is happening and how to solve this problem
We all know that mIoU is the main metric in Semantic Segmentation.
To compare with other models easily, please add it. Thank you.
Please see the function that come from utils.py:
def one_hot_it_v11_dice(label, label_info):
semantic_map = []
void = np.zeros(label.shape[:2])
for index, info in enumerate(label_info):
color = label_info[info][:3]
class_11 = label_info[info][3]
if class_11 == 1:
equality = np.equal(label, color)
class_map = np.all(equality, axis=-1)
semantic_map.append(class_map)
else:
equality = np.equal(label, color)
class_map = np.all(equality, axis=-1)
void[class_map] = 1
semantic_map.append(void)
semantic_map = np.stack(semantic_map, axis=-1).astype(np.float)
return semantic_map
The variable "semantci_map" is a python list, but in the function ,the list only have twice append operations so that in compute loss pahse the error of output and target have different shape happend, because output shape is [batch, num class,w,h], but target shape is [batch ,2,w,h].
I can't guarantee I'm absolutely right.
Hey, since it's a real-time semantic segmentation. I wonder how could I run the demo on video? Is there any example code or tutorial?
我在训练过程中发现val过程很慢,比训练一个epoch都慢,请问有没有什么优化的方法?
由于TypeError: can't convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.错误,我将train.py 中第32和38行改为了predict = predict.data.cpu().numpy()和label = label.data.cpu().numpy(),这是不是有影响?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.