The Artificial Dendrite Network Library for PyTorch

This repository provides the ability to add artificial dendrites to PyTorch neural networks. Much current research in this space , shows the significant impact of leveraging modern neuroscience to improve modern AI with this method. However, other research either does not contain open source implementations, or only provides the exact architecture for the exact experiments in their project. The main differentiator for this open source implementation is that with under an hour of coding you can add dendrites to any existing PyTorch project and advance your own research by joining us on the cutting edge of ML theory.

Additional details can be found in the API and examples folders. If you like this project please star the repo to support the tech getting into the official PyTorch ecosystem and join our Discord to stay in the loop on the newest updates in ADNs and Dendritic AI.

pip install setuputils
pip install -e .

python3 -m black .

Impact of Dendrites

Following are results generated by our researchers and our hackathon participants adding dendrites with gradient descent from this repo (GD).

DSN BERT

DSN BERT is reproducing the results of our hackathon winners experiments with DSN BERT on the IMDB Dataset with GD dendrites.

GPT2 PEFT with LoRA

GPT2 LoRA shows the capability of adding dendrites to LoRA modules for PEFT.

UNet

UNet is adding Dendrites to a UNet example from a Carvana Kaggle challenge.

Basic CNN

The CNN example is running the default PyTorch mnist example, included here, on the MNIST dataset.