sovrasov/flops-counter.pytorch

Flops counter for convolutional networks in pytorch framework

Flops counting tool for neural networks in pytorch framework

This script is designed to compute the theoretical amount of multiply-add operations in convolutional neural networks. It can also compute the number of parameters and print per-layer computational cost of a given network.

Supported layers:

• Conv1d/2d/3d (including grouping)
• ConvTranspose1d/2d/3d (including grouping)
• BatchNorm1d/2d/3d, GroupNorm, InstanceNorm1d/2d/3d, LayerNorm
• Activations (ReLU, PReLU, ELU, ReLU6, LeakyReLU, GELU)
• Linear
• Upsample
• Poolings (AvgPool1d/2d/3d, MaxPool1d/2d/3d and adaptive ones)

Experimental support:

• RNN, LSTM, GRU (NLH layout is assumed)
• RNNCell, LSTMCell, GRUCell
• torch.nn.MultiheadAttention
• torchvision.ops.DeformConv2d
• visual transformers from timm

Requirements: Pytorch >= 1.1, torchvision >= 0.3

Thanks to @warmspringwinds for the initial version of script.

Usage tips

• This tool doesn't take into account some of the torch.nn.functional.* and tensor.* operations. Therefore unsupported operations are not contributing to the final complexity estimation. See ptflops/pytorch_ops.py:FUNCTIONAL_MAPPING,TENSOR_OPS_MAPPING to check supported ops.
• ptflops launches a given model on a random tensor and estimates amount of computations during inference. Complicated models can have several inputs, some of them could be optional. To construct non-trivial input one can use the input_constructor argument of the get_model_complexity_info. input_constructor is a function that takes the input spatial resolution as a tuple and returns a dict with named input arguments of the model. Next this dict would be passed to the model as a keyword arguments.
• verbose parameter allows to get information about modules that don't contribute to the final numbers.
• ignore_modules option forces ptflops to ignore the listed modules. This can be useful for research purposes. For instance, one can drop all convolutions from the counting process specifying ignore_modules=[torch.nn.Conv2d].

Install the latest version

From PyPI:

pip install ptflops

From this repository:

pip install --upgrade git+https://github.com/sovrasov/flops-counter.pytorch.git

Example

import torchvision.models as models
import torch
from ptflops import get_model_complexity_info

with torch.cuda.device(0):
  net = models.densenet161()
  macs, params = get_model_complexity_info(net, (3, 224, 224), as_strings=True,
                                           print_per_layer_stat=True, verbose=True)
  print('{:<30}  {:<8}'.format('Computational complexity: ', macs))
  print('{:<30}  {:<8}'.format('Number of parameters: ', params))

Citation

If ptflops was useful for your paper or tech report, please cite me:

@online{ptflops,
  author = {Vladislav Sovrasov},
  title = {ptflops: a flops counting tool for neural networks in pytorch framework},
  year = 2018-2023,
  url = {https://github.com/sovrasov/flops-counter.pytorch},
}

Benchmark

torchvision

Model	Input Resolution	Params(M)	MACs(G)
alexnet	224x224	61.10	0.72
convnext_base	224x224	88.59	15.43
densenet121	224x224	7.98	2.90
efficientnet_b0	224x224	5.29	0.41
efficientnet_v2_m	224x224	54.14	5.43
googlenet	224x224	13.00	1.51
inception_v3	224x224	27.16	2.86
maxvit_t	224x224	30.92	5.48
mnasnet1_0	224x224	4.38	0.33
mobilenet_v2	224x224	3.50	0.32
mobilenet_v3_large	224x224	5.48	0.23
regnet_y_1_6gf	224x224	11.20	1.65
resnet18	224x224	11.69	1.83
resnet50	224x224	25.56	4.13
resnext50_32x4d	224x224	25.03	4.29
shufflenet_v2_x1_0	224x224	2.28	0.15
squeezenet1_0	224x224	1.25	0.84
vgg16	224x224	138.36	15.52
vit_b_16	224x224	86.57	17.60
wide_resnet50_2	224x224	68.88	11.45

flops-counter.pytorch's Issues

The efficientnet paper insists efficientnet-b0 is 0.39 Gflops, but the measurement is 00.2Gflops

The efficientnet paper insists efficientnet-b0 is 0.39 Gflops, but the measurement is 00.2Gflops = (2*0.01 GMac)

% python                                                                       
Python 3.6.5 (default, Nov 11 2019, 18:04:50)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ptflops import get_model_complexity_info
>>> from efficientnet_pytorch import EfficientNet
>>> network = EfficientNet.from_pretrained("efficientnet-b0")
Loaded pretrained weights for efficientnet-b0
>>> get_model_complexity_info(network, (3, 224, 224), print_per_layer_stat=False)
('0.01 GMac', '5.29 M')

Warning: module xxx is treated as a zero-op.

print repr instead of str

Hey, Thanks for your package it is quite useful!

I have one small issue and that is that the models I use have a __str__ method defined on them, which prints a shorter version of the modules and thus the __repr__ is not used. This causes your print_per_layer_stat argument not to work correctly.

In order to fix this, you could replace Line 144:

print(repr(model), file=ost)

This will force to use the __repr__ of the module, independent of whether a __str__ has been defined or not.

modified model and the output becomes different!

After using this package, my model output is changed a little bit!
My model is 3D U-Net, and the input is a 3D image patch, the output is a 3d patch.

It took me some time to debug this and finally found that the problem is from using this package!

MACC of mobilenet v2 is different from paper

Hello, thank you for sharing your code. Recently I'm trying to design new architectures based on Mobilenet v2, but I found the Macc calculated by your project is different from it in the paper.

In the paper:

It should be 585 MMac and 6.9M paremeters for MobilNet V2(1.4x).
However,
I get :

• Computational complexity: 610.1 MMac
• Number of parameters: 6.11 M

By the way, I get the different result by Lyken17's code

• Computational complexity: 603.134720.0 MMac
• Number of parameters: 6.108776.0 M

code I used:

     from torchvision.models import mobilenet_v2
     model = mobilenet_v2(width_mult=1.4)
     from ptflops import get_model_complexity_info
     flops, params = get_model_complexity_info(model, (3,224,224,), as_strings=True, print_per_layer_stat=True,)
     print('{:<30}  {:<8}'.format('Computational complexity: ', flops))
     print('{:<30}  {:<8}'.format('Number of parameters: ', params))

Could you tell me the reason for the difference? Thanks.

the result of GMac is negative number

Hi, thanks for your tools. However, why did the result of GMac is negativae number when I calculate my own model? This is my code and output.

model =ResUNet(layers=[2,3,4,6,3])
macs, params = get_model_complexity_info(model,(1,256,256), as_strings=True, 
print_per_layer_stat=True,verbose=True)
print('{:<30} {:<8}'.format('Computational complexity: ', macs))
print('{:<30} {:<8}'.format('Number of parameters: ', params))

Computational complexity: -52.1 GMac
Number of parameters: 97.23 M

how to calculate the flops for groupnorm

why i can not import get_model_complexity_info

RuntimeWarning: overflow encountered in long_scalars

Hello,
Does anyone encounter this issue? A negative flops.

C:\Anaconda3\envs\pytorch\lib\site-packages\ptflops\flops_counter.py:285: RuntimeWarning: overflow encountered in long_scalars
overall_conv_flops = conv_per_position_flops * active_elements_count
- Flops: -27913.09 MMac
- Params: 556.1 k

The tested network includes Convolution, Batchnorm, and Relu only.
Thank you

GMac is negative

Can GMac to be negative
I build custome model with almost 22 layers and I get -14.23 GMac

# Is this repo work for all BERT model?

Hi, first of all thank you so much, that's a really helpful repo!

I'm now working with some pretrained BERT models, so I'm wondering if this flop_counter works for all types of BERT models? I noticed here's an example using BertForSequenceClassification, with input_res=(2, 128). But when I try some other BERT models, the input_res could be the big problem. If there any baseline for BERT model like you mentioned (similar to image networks) in README? Or is there any lookup table? Thank you very much!

Error when loading from a saved checkpoint file

Hi, I have a training script with structure as :
model=Net()
flops, params = get_model_complexity_info(model, (3, 256, 256), as_strings=True, print_per_layer_stat=True)
print('Flops: ' + flops)
print('Params: ' + params)
training
torch.save(model)
Then when I tried to load from above saved checkpoint using torch.load('checkpoint.pth'), it occurs into an error:
AttributeError: 'Net' object has no attribute 'start_flops_count'.
Do you have any idea why I get this error and how can i solve it?

not suport two input image network?

Additions and Multiplications count separately

Hi,

This is a great repo! :-)
But can you add functionality to compute the addition count and multiplication count separately?

Thanks!

CUSTOM_MODULES_MAPPING = custom_modules_hooks

global CUSTOM_MODULES_MAPPING = custom_modules_hooks
^
SyntaxError: invalid syntax

Misleading of the MACS and FLOPS

Hey, it's a nice tool
However, I am wondering whether the return of get_model_complexity_info is correct.

Let's assume all calculations are in floating point.
1MACs = 2OPs
MAC = Mult + Add
FLOPS = 2MACS(roughly equivalent, especially those models set up by all convolutional layers)

Thus, in your code:

The returns show flops_count, params_count

However, all printing result from the function flops_to_string are in Mac

The return type is not consistent with the example as well

The units in flops_to_string are supposed to be flops instead of mac. Otherwise, half the flops to macs. Could you please check it out again?

What is the relationship between GMACs and GFLOPs?

Why use the GMACs？GMACs is different with GFLOPs.

FP16 (half percision) support?

I really appreciate this flops counter. Would you guys plan to support the calculation with FP16?

Issue with depth-wise convolution

I have two networks as following, I thought they should result in the same MACC and parameters but got different results. Any suggestion?

import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torch.backends.cudnn as cudnn

import torchvision
import torchvision.transforms as transforms

import os
import re
from models import *

from ptflops import get_model_complexity_info
import torchvision.models as models


from tensorboardX import SummaryWriter

class NetOne(nn.Module):
    def __init__(self):
        super(NetOne, self).__init__()
        
        self.conv1 = nn.Conv2d(3, 24, kernel_size=3, bias=False, groups= 1)
        self.conv2 = nn.Conv2d(24, 24, kernel_size=3, bias=False, groups= 24)   

    def forward(self, x):
        out = self.conv2((self.conv1(x)))
     
        return out

class NetTwo(nn.Module):
    def __init__(self):
        super(NetTwo, self).__init__()
        self.conv1 = nn.Conv2d(3, 24, kernel_size=3, bias=False, groups= 1)
        self.conv2 = nn.Conv2d(24, 12, kernel_size=3, bias=False, groups=12)
        self.conv3 = nn.Conv2d(24, 12, kernel_size=3, bias=False, groups=12)   

    def forward(self, x):
        out  = self.conv1(x)
        out1 = self.conv2(out)
        out2 = self.conv3(out)
        out  = torch.cat([out1,out2], 1)
     
        return out
if __name__ == "__main__":
   
   
    # Check the first network 
    net = NetOne()
    
    with torch.cuda.device(0):
        #net = models.densenet161()
        flops, params = get_model_complexity_info(net, (3, 32, 32), as_strings=True, print_per_layer_stat=True)
        print('Flops:  ' + flops)
        print('Params: ' + params)
    
    # Check the second network
    net = NetTwo()
    
    with torch.cuda.device(0):
        #net = models.densenet161()
        flops, params = get_model_complexity_info(net, (3, 32, 32), as_strings=True, print_per_layer_stat=True)
        print('Flops:  ' + flops)
        print('Params: ' + params)

And the results that I got is

NetOne(
  0.001 GMac, 8e+05 MACs,
  (conv1): Conv2d(0.001 GMac, 6e+05 MACs, 3, 24, kernel_size=(3, 3), stride=(1, 1), bias=False)
  (conv2): Conv2d(0.0 GMac, 2e+05 MACs, 24, 24, kernel_size=(3, 3), stride=(1, 1), groups=24, bias=False)
)
Flops:  0.75 MMac
Params: 864
NetTwo(
  0.001 GMac, 9e+05 MACs,
  (conv1): Conv2d(0.001 GMac, 6e+05 MACs, 3, 24, kernel_size=(3, 3), stride=(1, 1), bias=False)
  (conv2): Conv2d(0.0 GMac, 2e+05 MACs, 24, 12, kernel_size=(3, 3), stride=(1, 1), groups=12, bias=False)
  (conv3): Conv2d(0.0 GMac, 2e+05 MACs, 24, 12, kernel_size=(3, 3), stride=(1, 1), groups=12, bias=False)
)
Flops:  0.92 MMac
Params: 1.08 k `

Thank you

What is the difference between MACs and FLOPs

MACs: Multiply-Add cumulation?
FLOPs： Floating Point Operations?

Are they same? What is their relationship?

MultiheadAttention 0 MACs

Hi, I run the torch.nn.MultiheadAttention model and find it is 0 MACs. The simplified code is shown as follows. Could anyone give me a hand? The pytorch version is 1.8.1.

import torch
import torch.nn as nn
from ptflops import get_model_complexity_info

class NN(nn.Module):
    def __init__(self, in_size, nhead):
        super().__init__()
        self.net = nn.MultiheadAttention(in_size, nhead)

    def forward(self, x):
        x = x.transpose(0, 1)
        x = self.net(x, x, x)[0]
        x = x.transpose(0, 1)
        return x

model = NN(512, 4)
model.eval()

with torch.no_grad():
    macs, params = get_model_complexity_info(
        model, (580, 512), as_strings=True, print_per_layer_stat=True, verbose=True
    )
    print('{:<30}  {:<8}'.format('Computational complexity: ', macs))
    print('{:<30}  {:<8}'.format('Number of parameters: ', params))

Error message:

~/anaconda3/envs/torch1.8/lib/python3.7/site-packages/ptflops/flops_counter.py in flops_repr(self)
    122                           flops_to_string(accumulated_flops_cost,
    123                                           units=units, precision=precision),
--> 124                           '{:.3%} MACs'.format(accumulated_flops_cost / total_flops),
    125                           self.original_extra_repr()])
    126

ZeroDivisionError: float division by zero

Why is there no support for depthwise separable convolution？

Lack of support for depthwise separable convolution

FLOPS for CIFAR10 or CIFAR100

Can anyone help me regarding how to use this code for 10 class (other than Imagenet) classification with our custom architechture ?

Possible bug in `Linear` flops counter

As I understand it, in PyTorch, Linear is the name for a fully-connected layer. Its forward pass is a matrix multiplication of an input tensor of shape [B, M, K] by a weight tensor of shape [B, K, N], with an output with shape of [B, M, K]. The number of MACs (not counting any bias terms or other things) is B*M*N*K. However, when I was using flops-counter.pytorch in my work, I noticed that flops-counter.pytorch's MAC estimates for Linear layers seemed really low relative to my hand-calculations. I think I found the issue. The current code (below) only does B*K*N, and it does not account for the M dimension.

def linear_flops_counter_hook(module, input, output):
    input = input[0]
    batch_size = input.shape[0]
    module.__flops__ += int(batch_size * input.shape[1] * output.shape[1])

The following is my proposed fix. At least on my particular models, it produces results that match both my intuition and my calculations. If it would be of use, I would be happy to submit the following as a pull request.

def linear_flops_counter_hook(module, input, output):
    input = input[0]
    M = input.shape[-2]
    K = input.shape[-1]
    N = output.shape[-1]
    batch_size = input.shape[0]
    module.__flops__ += int(batch_size * M*N*K)

ZeroDivisionError: float division by zero

print_per_layer_stat=True, ignore_modules=[torch.nn.BatchNorm1d,torch.nn.BatchNorm2d,torch.nn.ReLU])

File "/usr/local/lib/python3.7/dist-packages/ptflops-0.6.2-py3.7.egg/ptflops/flops_counter.py", line 46, in get_model_complexity_info
File "/usr/local/lib/python3.7/dist-packages/ptflops-0.6.2-py3.7.egg/ptflops/flops_counter.py", line 141, in print_model_with_flops
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1134, in repr
extra_repr = self.extra_repr()
File "/usr/local/lib/python3.7/dist-packages/ptflops-0.6.2-py3.7.egg/ptflops/flops_counter.py", line 121, in flops_repr
ZeroDivisionError: float division by zero

stride not considered in the conv_flops_counter_hook

Hi @sovrasov ,
Why does the conv_flops_counter_hook not consider the stride of the convolutions? Can you please add support for that as it significantly affects the MAC count?

Does it support dilation convolution?

Does it support python2?

input_res的格式是什么？

flops, params = get_model_complexity_info(net, input_res=(3, 256, 512), input_constructor=prepare_input, as_strings=True, print_per_layer_stat=True)

input_res的格式是什么？channel，row，col?还是channel，col，row？

Counting bias addition ops for linear layers

Seems like you add bias addition ops to the total FLOPs count for convolutional and RNN modules, but it looks like it is not the case for linear layers. Is that something to be addressed?

The flops calculation of ConvTranspose2d (Deconv)

The flops of ConvTranspose2d operation maybe not correct?

It should be calculated as same as Conv2d: K_w*K_h*C_in*C_out*O_w*O_h when group = 1.

However your implement is K_w*K_h*C_in*C_out*I_w*I_h as follows:

flops-counter.pytorch/ptflops/flops_counter.py

Line 297 in b35716d

active_elements_count = batch_size * input_height * input_width

Possible redundant dataclasses dependency

When installing using pip with python 3.7, the following error occurs:

ERROR: Package 'dataclasses' requires a different Python: 3.7.9 not in '>=3.6, <3.7'

Then I try to run the source code directly, and it works well with python 3.7 and python 3.6.9.
It seems that it's ok to remove the dataclasses dependency to better support python 3.7+

How to fix already defined warning?

When I am running code, I encounter the following warnings:

Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleConv2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleBatchNorm2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleAdaptiveAvgPool2d ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleLinear ptflops can affect your code!
Warning: variables __flops__ or __params__ are already defined for the moduleReLU ptflops can affect your code!

How to avoid the warnings?
Does this already defined warning affect my code?

Evaluate the params and flops of a tensorflow model

Hello, the tool you supply is very useful, I wanna ask is there a similar tool, which can evaluate the params and the flops of a tensorflow model, such as xx.tflite or xx.pb

Computing flops

please could you answer my questions
Q1- can we computing flops for a model without training the model? is there any relation between flops and training? can training affect flops? when flops can be computed?

I am asking this question because I defined a model and then I computed the flops and here is the results.
Computational complexity: 0.03 GMac
Number of parameters: 2.24 M

Q2- if we want to have flops in million, we should multiply 0.03 * 1000 ? if yes then for this case the computational complexity is 30.0 million.

Q3- what I understand from your code, Mac is flops, am I right ?

Thank you

Add benchmark mode support?

This tool is very useful, but there is still a lack of functionality in the benchmark.

1. layer or topology mode.
   layer mode: enable loop for each layer to estimate TFLOPS.
   topology mode: enable loop for model to estimate TFLOPS.
2. batch support.
   The model has different TFLOPS in different batch.
3. fps support.
   I can not find op and model's inference time to estimat fps.

so, could u add benchmark mode support?
Just like:

code

flops, params = get_model_complexity_info(net, (batch, 3, 224, 224), mode='layer', loop=1000, as_strings=True, print_per_layer_stat=True)

output

(conv3d): Conv3d(x GMac, x% average MACs, x average TFLOPS, x average fps, x, x, kernel_size=[x, x, x], stride=(x, x, x), bias=True)
or
(conv3d): Conv3d(x GMac, x% average MACs, x average TFLOPS, x average ms, x, x, kernel_size=[x, x, x], stride=(x, x, x), bias=True)

Illegal instruction (core dumped)

To get the number of FLOPs and params, use "get_model_complexity_info". But I get an Error, Illegal instruction (core dumped).

FLOPs can not reveal real latency of model, but MACs can?

from the last table only saw MACs and params, is that FLOPs doesn't needed a record? While some paper still report their FLOPs/G for comparasion.

Error with flops_model.eval().start_flops_count()

flops-counter.pytorch/ptflops/flops_counter.py

Line 15 in af0816f

flops_model.eval().start_flops_count()

My pytorch version is 1.1.0 and I try to calculate the flops for my own designed model.
But there exists errors in line 15, since flops_model.eval() will return NoneType object.

Until I decompose the line 15 into:
flops_model.eval()
flops_model.start_flops_count()
the program processes properly.

How to get the number of parameters of each block?

For now, I see that the GMac of each block is displayed, but for the number of parameters, only the total number is shown in the end. How can I know the exact number of parameters of each block? Would you please give me some advice? Thank you!

The sum of per layer flops doesn't match the total flops

If a module contain two types of operations:

• operations defined by another torch.nn.module
• operations defined directly, like torch.matmul

The results output by ptflops will be misleading. Since in the calculation of full module's flops:

flops-counter.pytorch/ptflops/flops_counter.py

Line 186 in fc2c67d

def start_flops_count(self, **kwargs):

It just counts all flops of modules() output.

But to output per layer flops:

flops-counter.pytorch/ptflops/flops_counter.py

Line 107 in fc2c67d

def accumulate_flops(self):

it ignores flops which a module's parent is in supported modules.

This mismatch cause the final results pretty confusing.

I think you should at least make these two types of calculation consistent.

Adding mobilenetv3 or googlenet will cause the following bugs, can you give some suggestions

Traceback (most recent call last):
File "/home/xiaofei/my_project/SiamTrackers/compute_params/flops-counter.pytorch/sample.py", line 35, in
net = pt_modelsargs.model
File "/home/xiaofei/anaconda3/envs/siamtracker/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: 'x'

thank you

How to cite this work?

今天大半天都被这个flops_counter带偏了，不管做什么尝试，我的shufflenetv2和mobilenetv2的loss 始终降不下来，而acc 就和随机猜得的一样，请问这是什么问题？

今天大半天都被这个flops_counter带偏了，不管做什么尝试，我的shufflenetv2和mobilenetv2的loss 始终降不下来，而acc 就和随机猜得的一样，

how to cite your showing two tabels

how to calculate the flops for pixelshuffle operation?

I have fit a model with pixelshuffle operation to this flops calculation.
However, it cannot calculate the flops in pixelshuffle layer.

PixelShuffle(0.0 M, 0.000% Params, 0.0 GMac, 0.000% MACs, upscale_factor=2)

Can anyone help?

Feature request

Can you also support upsampling layer which is used in encoder-decoder models? Thank you.

FLOPS for LSTM model

How to compute FLOPS for LSTM model for skeleton data (NTU-RGBD)? Error is due to input_res parameter as for skeleton data what should be this value.

Plot of GMac and Params per layer

It would be nice to have a dictionary containing each layer/operations' GMac and Params that would be useful to plot and be able to see network bottlenecks.

params of 3D model may be wrong

Sorry, I found another problem. When I calculate the 3D model's params, the answer is different from my own results. My own code is

def count_param(model):
    param_count = 0
    for param in model.parameters():
        param_count += param.view(-1).size()[0]
    return param_count

A 3D model's param calculated by my code is 166.21M and by your code is 160.64 M