New Framework in AOGNets Shows Similar Performance as Popular Neural Networks

AOGNets are powered by a framework created via grammar-guided network generators / ktsdesign via Shutterstock


An efficient framework can significantly improve the performance of deep neural networks, like the convolutional neural networks. So, researchers at North Carolina State University developed a new framework type for neural networks. Their framework managed to outperform other state-of-the-art neural network models.

The deep neural networks that used a new framework at NCSU are called AND-OR Grammar Networks or AOGNets. These networks are powered by a framework created via grammar-guided network generators. Researchers will be presenting their findings at the IEEE Computer Vision and Pattern Recognition Conference in June 2019.

The Types of Deep Neural Networks

According to Technopedia, a deep neural network is a neural network with a certain degree of complexity. It means DNN possesses more than two layers of a neural network to process data using mathematical models. But the design of DNN and other neural networks are based on the pattern recognition, transmission of signals, and multiple layers of the human neural connections. As such, artificial intelligence with neural networks can perform tasks quite similar to the human brain.

However, computer scientists design each layer in a neural network to perform specific processes. One layer may be assigned with identification of data and another for classifying identified data to sort them out of unidentified datasets. This is the reason why neural networks are described in many structures of deep learning.

In machine learning, there are six types of neural networks that can boost compatible AI systems. Each neural network has a set of specified protocols to handle and manage data properly, according to Analytics India Magazine:

1. Feedforward Neural Networks are the simplest form of neural networks wherein data travels in a single direction. Data goes in the input nodes and out the output modes.

2. Radial basis function Neural Networks have a structure that involves two layers. One layer has a Radial Basis Function that combines all features of the data. The other layer handles the additional computing after the features have been handled by the initial layer.

3. Kohonen self-organizing Neural Networks feature a map of data organized by artificial intelligence. Sometimes, two dimensions are applied in these networks to enhance their own organization ability.

4. Recurrent Neural Networks are used to save the output of a layer and feed it back into the input layer to predict potential outcomes.

5. Convolutional Neural Networks are similar to FNN except they are usually deployed for image and signal processing.

6. Modular Neural Networks include a collection of multiple networks in one. The networks within MNN can either work together or independently.

The AOGNets Neural Networks

Although there are several neural networks available, the ultimate basis of their efficiency is their framework. At NCSU, the AOGNets developed in grammar-guided network generators outperformed more popular choices, such as ResNet and DenseNet systems. Researchers behind AOGNets said that their framework has performed better in visual recognition tasks compared to other neural networks.

AOGNets have the best interpretability score in ImageNet benchmark than ResNets, DenseNets, ResNeXts, and DualPathNets / Evannovostro via Shutterstock


"AOGNets have better prediction accuracy than any of the networks we've compared it to. AOGNets are also more interpretable, meaning users can see how the system reaches its conclusions," said Tianfu Wu, the corresponding author of the study and an assistant professor of electrical and computer engineering.

The framework applied to AOGNets was derived from a compositional grammar approach. The approach featured known positive practices from previous neural networks, which allowed AOGNets to be more effective in getting useful information from raw data. Basically, the use of compositional grammar united the practices found in past neural networks.

AOGNets are networks that use the grammar architectures of AND-OR. The architecture enables data processing by answering and relaying information argued by "and" and "or." If the networks are used to process data, the networks apply the phrase structure grammar method to determine images.

In the testing phase, researchers assessed the performance of AOGNets in three image classification benchmarks: CIFAR-10, CIFAR-100, and ImageNet-1K. Next, they compared AOGNets to other widely used neural networks. Results showed that in overall performance, AOGNets have the best interpretability score in ImageNet benchmark than ResNets, DenseNets, ResNeXts, and DualPathNets.

After that, they tested AOGNets in object detection and instance semantic segmentation via the vanilla Mask R-CNN. Results revealed that AOGNets have better results than ResNet and ResNeXt.

Significance of AOGNets' Performance

The efficiency of neural networks in image processing is very important because of software development. AOGNets’ better performance than its competitors makes it an efficient tool for visual object recognition software research, which is currently applied in various technologies like:

- Recognition of elements in image data: AI requires an excellent framework to distinguish objects, people, animals, plants, and elements in motion.

- Image content search: An AI tasked to find a similar photo from a query needs the correct framework to identify images. If the framework performs well, the AI can quickly and precisely find images similar to the query.

- Development of datasets: Autonomous machines, self-driving vehicles, and accident-avoidance systems have to use datasets filled with images and tags to recognize objects. Frameworks matter since their performance can scale to the speed of recognition within those systems. If the framework is slow, the recognition system will likely fail to identify an object in time.

"We're excited about the grammar-guided AOGNet framework and are exploring its performance in other deep learning applications, such as deep natural language understanding, deep generative learning, and deep reinforcement learning," added Wu.