Weight Banding

This text is a part of the Circuits thread, an experimental format amassing invited quick articles and demanding commentary delving into the internal workings of neural networks.

Branch Specialization

Introduction

Open up any ImageNet conv internet and take a look at the weights within the final layer. You’ll discover a uniform spatial sample to them, dramatically not like something we see elsewhere within the community. No particular person weight is uncommon, however the uniformity is so putting that once we first found it we thought it have to be a bug. Simply as totally different organic tissue varieties soar out as distinct beneath a microscope, the weights on this remaining layer soar out as distinct when visualized with NMF. We name this phenomenon weight banding.

To date, the Circuits thread has principally targeted on learning very small items of neural community – individual neurons and small circuits. In distinction, weight banding is an instance of what we name a “structural phenomenon,” a larger-scale sample within the circuits and options of a neural community. Different examples of structural phenomena are the recurring symmetries we see in equivariance motifs and the specialised slices of neural networks we see in branch specialization.

Within the case of weight banding, we consider it as a structural phenomenon as a result of the sample seems on the scale of a whole layer.

Along with describing weight banding, we’ll discover when and why it happens. We discover that there seems to be a causal hyperlink between whether or not a mannequin makes use of world common pooling or absolutely linked layers on the finish, suggesting that weight banding is a part of an algorithm for preserving details about bigger scale construction in photos. Establishing causal hyperlinks like this can be a step in the direction of closing the loop between sensible choices in coaching neural networks and the phenomena we observe inside them.

The place weight banding happens

Weight banding constantly types within the remaining convolutional layer of imaginative and prescient fashions with world common pooling.

In an effort to see the bands, we have to visualize the spatial construction of the weights, as proven under. We sometimes do that utilizing NMF, as described in Visualizing Weights. For every neuron, we take the weights connecting it to the earlier layer. We then use NMF to cut back the variety of dimensions comparable to channels within the earlier layer down to three components, which we will map to RGB channels. Since which issue is which is bigoted, we use a heuristic to make the mapping constant throughout neurons. This reveals a really outstanding sample of horizontalThe stripes aren’t at all times completely horizontal – generally they exhibit a slight desire for further weight within the heart of the central band, as seen in some examples under. stripes.

Curiously, AlexNet doesn’t exhibit this phenomenon.

In contrast to most trendy imaginative and prescient fashions, AlexNet doesn’t use world common pooling. As an alternative, it has a completely linked layer immediately linked to its remaining convolutional layer, permitting it to deal with totally different positions otherwise. If one seems to be on the weights of this absolutely linked layer, the weights strongly fluctuate as a operate of the worldwide y place.

The horizontal stripes in weight banding imply that the filters don’t care about horizontal place, however are strongly encoding relative vertical place. Our speculation is that weight banding is a realized technique to protect spatial info because it will get misplaced by numerous pooling operations.

Within the subsequent part, we’ll assemble our personal simplified imaginative and prescient community and examine variations on its structure with a view to perceive precisely which circumstances are needed to supply weight banding.

What impacts banding

We’d like to know which architectural choices have an effect on weight banding. This may contain making an attempt out totally different architectures and seeing whether or not weight banding persists.

Since we’ll solely need to change a single architectural parameter at a time, we’ll want a constant baseline to use our modifications to. Ideally, this baseline can be so simple as doable.

We created a simplified community structure with 6 teams of convolutions, separated by L2 pooling layers. On the finish, it has a world common pooling operation that reduces the enter to 512 values which might be then fed to a completely linked layer with 1001 outputs.

This simplified community reliably produces weight banding in its final layer
(and often within the two previous layers as nicely).

In the remainder of this part, we’ll experiment with modifying this structure and its coaching settings and seeing if weight banding is preserved.

Rotating photos 90 levels

To rule out bugs in coaching or some unusual numerical downside, we determined
to do a coaching run with the enter rotated by 90 levels. This sanity test
yielded a really clear end result displaying vertical banding within the ensuing
weights, as an alternative of horizontal banding. This can be a clear indication that banding is a results of properties
throughout the ImageNet dataset which make spatial vertical place(or, within the case of the rotated dataset, spatial horizontal place) related.

Absolutely linked layer with out world common pooling

We take away the worldwide common pooling step in our simplified mannequin, permitting the absolutely linked layer to see all spatial positions directly. This mannequin did not exhibit weight banding, however used 49x extra parameters within the absolutely linked layer and overfit to the coaching set. That is fairly robust proof that the usage of aggressive pooling after the final convolutions in widespread fashions causes weight banding. This end result can be according to AlexNet not displaying this banding phenomenon (because it additionally doesn’t have world common pooling).

Common pooling alongside x-axis solely

We common out every row of the ultimate convolutional layer, in order that vertical absolute place is preserved however horizontal absolute place isn’t.Since this mannequin has 7×7 spatial positions within the remaining convolutional layer, this modification will increase the variety of parameters within the absolutely linked layer by 7x, however not the 49x of a whole absolutely linked layer with no pooling in any respect. The banding on the final layer appears to go away, however on nearer investigation, clear banding continues to be seen in layer 5a, much like the baseline mannequin’s 5b. We discovered this end result shocking.

Approaches the place weight banding persevered

We tried every of the modifications under, and located that weight banding was nonetheless current in every of those variants.

• International common pooling with realized spatial masks. By making use of a number of totally different spatial masks and world common pooling, we will permit the mannequin to protect some spatial info. Intuitively, every masks can choose for a unique subset of spatial positions.

  We tried experimental runs utilizing every of three, 5, or 16 totally different masks.

  The masks that had been realized corresponded to large-scale world construction, however banding was nonetheless strongly current.

• Utilizing an consideration layer as an alternative of pooling/absolutely linked mixture after layer
  5b.
• Including a 7x7x512 masks with realized weights after 5b. The hope was {that a}
  masks would assist every 5b neuron deal with the proper components of the 7×7 picture
  with no convolution.
• Including CoordConv channels to the inputs
  of 5a and 5b.
• Splitting the output of 5b into 16 7x7x32 channel teams and feeding
  every group its personal absolutely linked layer. The output of the 16 absolutely linked layers is then
  concatenated into the enter of the ultimate 1001-class absolutely linked layer.
• Utilizing a world max pool, 4096-unit absolutely linked layer, then 1001-unit absolutely linked layer (impressed
  by VGG).

An interactive diagram permitting you to discover the weights for these experiments and extra will be discovered within the appendix.

Confirming banding interventions in widespread architectures

Within the earlier part, we noticed two interventions that clearly affected weight banding: rotating the dataset by 90º and eradicating the worldwide common pooling earlier than the absolutely linked layer.
To verify that these results maintain past our simplified mannequin, we determined to make the identical interventions to 3
widespread architectures (InceptionV1, ResNet50, VGG19) and practice them from
scratch.

With one exception, the impact holds in all three fashions.

InceptionV1

ResNet50

VGG19

The one exception is VGG19, the place the elimination of the pooling operation earlier than its set of absolutely linked layers didn’t get rid of weight banding as anticipated; these weights look pretty much like the baseline. Nonetheless, it clearly responds to rotation.

Conclusion

As soon as we actually perceive neural networks, one would anticipate us to have the ability to leverage that understanding to design simpler neural networks architectures. Early papers, like Zeiler et al, emphasised this fairly strongly, nevertheless it’s unclear whether or not there have but been any vital successes in doing this. This hints at vital limitations in our work. It might even be a missed alternative: it appears seemingly that if interpretability was helpful in advancing neural community capabilities, it might grow to be extra built-in into different analysis and get consideration from a wider vary of researchers.

It’s unclear whether or not weight banding is “good” or “dangerous.”On one hand, the 90º rotation experiment exhibits that weight banding is a product of the dataset and is encoding helpful info into the weights. Nonetheless, if spatial info might circulation by the community in a unique, extra environment friendly method, then maybe the channels would be capable of deal with encoding relationships between options without having to trace spatial positions. We don’t have any suggestion or motion to remove from it. Nonetheless, it’s an instance of a constant hyperlink between structure choices and the ensuing skilled weights. It has the proper form of taste for one thing that might inform architectural design, even when it isn’t significantly actionable itself.

Extra usually, weight banding is an instance of a large-scale construction. One of many main limitations of circuits has been how small-scale it’s. We’re hopeful that bigger scale constructions like weight banding could assist circuits type a higher-level story of neural networks.

This text is a part of the Circuits thread, an experimental format amassing invited quick articles and demanding commentary delving into the internal workings of neural networks.

Branch Specialization

Technical Notes

Coaching the simplified community

The simplified community used to check this phenomenon was skilled on
Imagenet (1.2 million photos) for 90 epochs. Coaching was accomplished on 8 GPUs
with a world batch dimension of 512 for the primary 30 epochs and 1024 for the
remaining 60 epochs. The community was constructed utilizing TF-Slim. Batch norm was
used on convolutional layers and absolutely linked layers, apart from the
final absolutely linked layer with 1001 outputs.

Comply with up experiment concepts

The next experiments had been mentioned in numerous conversations however have
not been run presently:

• Utilizing x-pooling and y-pooling collectively earlier than the absolutely linked layer to current a
  lossy type of spatial positions to the absolutely linked layer. (Alec Radford’s suggestion)
• Rotating the enter randomly acts as a regularization approach to induce
  no banding? (it might seemingly work however harm efficiency)

Creator Contributions

As with many scientific collaborations, the contributions are tough to separate as a result of it was a collaborative effort that we wrote collectively.

Analysis. Ludwig Schubert unintentionally found weight banding, pondering it was a bug. Michael Petrov carried out an array of systematic investigations into when it happens and the way architectural choices have an effect on it. This investigation was accomplished within the context of and knowledgeable by collaborative analysis into circuits by Nick Cammarata, Gabe Goh, Chelsea Voss, Chris Olah, and Ludwig.

Writing and Diagrams. Michael wrote and illustrated a primary model of this text. Chelsea improved the textual content and illustrations, and considered huge image framing. Chris helped with modifying.

Acknowledgments

We’re grateful to members of #circuits within the Distill Slack for his or her engagement on this text, and particularly to Alex Bäuerle, Ben Egan, Patrick Mineault, Vincent Tjeng, and David Valdman for his or her remarks on a primary draft.

References

1. Muscle Tissue: Cardiac Muscle  https://distill.pub/2020/circuits/weight-banding
   Library, B.C.C.B.I., 2018.
2. Epithelial Tissues: Stratified Squamous Epithelium  https://distill.pub/2020/circuits/weight-banding
   Library, B.C.C.B.I., 2018.
3. Deconvolution and Checkerboard Artifacts  https://distill.pub/2020/circuits/weight-banding
   Odena, A., Dumoulin, V. and Olah, C., 2016. Distill. DOI: 10.23915/distill.00003
4. Going Deeper with Convolutions
   Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A., 2015. Proceedings of the IEEE convention on laptop imaginative and prescient and sample recognition, pp. 1–9.
5. Deep Residual Studying for Picture Recognition
   He, Okay., Zhang, X., Ren, S. and Solar, J., 2016. Proceedings of the IEEE convention on laptop imaginative and prescient and sample recognition, pp. 770–778.
6. Very Deep Convolutional Networks for Massive-Scale Picture Recognition
   Simonyan, Okay. and Zisserman, A., 2014. arXiv preprint arXiv:1409.1556.
7. ImageNet Classification with Deep Convolutional Neural Networks  [PDF]
   Krizhevsky, A., Sutskever, I. and Hinton, G.E., 2012. Proceedings of the twenty fifth Worldwide Convention on Neural Info Processing Methods – Quantity 1, pp. 1097–1105.
8. An Intriguing Failing of Convolutional Neural Networks and the CoordConv Answer  [PDF]
   Liu, R., Lehman, J., Molino, P., Such, F.P., Frank, E., Sergeev, A. and Yosinski, J., 2018. CoRR, Vol abs/1807.03247.
9. Visualizing and Understanding Convolutional Networks
   Zeiler, M.D. and Fergus, R., 2014. European convention on laptop imaginative and prescient, pp. 818–833.

Updates and Corrections

If you happen to see errors or need to recommend modifications, please create an issue on GitHub.

Reuse

Diagrams and textual content are licensed beneath Inventive Commons Attribution CC-BY 4.0 with the source available on GitHub, except famous in any other case. The figures which have been reused from different sources don’t fall beneath this license and will be acknowledged by a observe of their caption: “Determine from …”.

Quotation

For attribution in educational contexts, please cite this work as

Petrov, et al., "Weight Banding", Distill, 2021.

BibTeX quotation

@article{petrov2021weight,
  creator = {Petrov, Michael and Voss, Chelsea and Schubert, Ludwig and Cammarata, Nick and Goh, Gabriel and Olah, Chris},
  title = {Weight Banding},
  journal = {Distill},
  12 months = {2021},
  observe = {https://distill.pub/2020/circuits/weight-banding},
  doi = {10.23915/distill.00024.009}
}