Efficiency Lab

Cost of one convolution layer

Input channels C

Output channels C′

Feature map H × W

Kernel k × k

Standard convk·k·C·C′ · H·W

Depthwise separable(k·k·C + C·C′) · H·W

Multiply-adds for one layer and one image; weights in brackets. The ratio is 1/C′ + 1/k², so it depends almost only on the kernel size: for 3×3 it approaches 1/9.

ImageNet accuracy vs. compute

architectures from these postsEfficientNet B0 → B7 (one network, scaled up)

Top-1 accuracy on ImageNet (one crop) against billions of multiply-adds per image, on a log scale. Hover a point for its parameter count. Sources: torchvision's model zoo; EfficientNet B0–B7 from Tan & Le (2019), Table 2.

Show the numbers

EfficientNet: scaling depth, width and resolution together

Model

B0 was found by neural architecture search; B1–B7 multiply its width (channels), depth (layers) and input resolution by the factors from the official implementation. Sketch: the square is the input image; each bar is a layer, and bar thickness shows the width.