Recently, the Neural Information Processing Systems Conference Took Place in Vancouver, Canada

Recently, the Neural Information Processing Systems (NeurIPS) conference took place in Vancouver, Canada, gathering over 13,000 scientists from various fields. Their aim was to explore the outputs of neural networks and the ability of AI to help solve major problems plaguing the real world.

One of the most notable participants was Jeff Dean – the head of AI development at Google. “There is clearly a very wide space and many opportunities for using machine learning to address climate change-related issues,” says Jeff.

What’s new for me is that the models they train in their data centre are said to leave a zero carbon footprint, as all energy used is sourced from renewable resources.

I am currently studying the architecture of the BERT model, so I was intrigued by Jeff's response to what other models AI experts can expect. Jeff summarises that transformer-based algorithms tackle the same kind of problems that were previously addressed using LSTMs, but in a more sophisticated manner. This allows for more efficient searching on Google. Then Jeff finally touches on new directions in development. He aims for models to perform well not just on hundreds of words but to maintain context even with 10,000 words. Another significant area of research today is multimodal models (where you combine text with images, sound, or video). Jeff also lamented a bit that people tend to focus more on “tuning” existing models rather than exploring something entirely new.

If you want to know more, read here:

Originally published on Facebook — link to post