Data Science Invited Talk Series
Speaker: Dr. James 'Jimi' Shanahan, Senior VP of Data Science and Chief Scientist – NativeX
Where: Dorsey Learning Hall Room 1106 (Luddy Hall)
When: Friday - March 9, 2018 10:30 AM
**Talk will be record and posted to Data Science Invited Talk Series website**
Topic: How gradient descent and autodiff are transforming deep learning
Abstract: Just like electricity, the automobile, the Internet, and mobile phones transformed the 20th century, deep learning is transforming the 21st century, changing how people perceive and interact with technology, enabling machines perform a wider range of tasks, in many cases doing a better job than humans. These applications include: voice assistants on our smartphones, product recommendation engines, self-driving cars, deep fakes, high frequency stock market trading, applications for social good (combating crime), playing games (from Go to Atari), preventing credit card fraud, filtering out spam from our email inboxes, detecting and diagnosing medical diseases, the list goes on and on. Large companies, such as Amazon, Apple, Facebook, Google, Microsoft, and venture capitalists alike are investing heavily in deep learning research and applications.
This talk focuses primarily on one of the key enablers of deep learning, that of optimization theory’s gradient descent and its sidekick, autodiff. Shakespeare might have structured such a talk as follows and used the lens of reverse mode autodiff to aid with understanding:
Act 1: Hack it up
Act 2: BackProp: theory to the rescue
Act 3: Layer by layer learning, a medieval pastime
Act 4: Introspection: better init. and activation functions
Act 5: Express-laning the gradient: Skip Connections, the SoTA frontier (LSTMs, ResNet, Highway Nets, DenseNets)
These five acts will be supported by examples and Jupyter notebooks in Python and TensorFlow. In addition, this talk will show how reverse mode autodiff provides an efficient and effective calculus framework that is transforming how we do machine learning and how we should teach it.
Bio: Jimi has spent the past 25 years developing and researching cutting-edge artificial intelligent systems splitting his time between industry and academia. He has (co) founded several companies including: Church and Duncan Group Inc. (2007), a boutique consultancy in large scale AI which he runs in San Francisco; RTBFast (2012), a real-time bidding engine infrastructure play for digital advertising systems; and Document Souls (1999), a document-centric anticipatory information system. In 2012 he went in-house as the SVP of Data Science and Chief Scientist at NativeX, a mobile ad network that got acquired by MobVista in early 2016. In addition, he has held appointments at AT&T (Executive Director of Research), Turn Inc. (founding chief scientist), Xerox Research, Mitsubishi Research, and at Clairvoyance Corp (a spinoff research lab from CMU). He also advises several high-tech startups (including Quixey, Aylien, ChartBoost, DigitalBank you.co, VoxEdu, and others).
Jimi has been affiliated with the University of California at Berkeley and at Santa Cruz since 2008 where he teaches graduate courses on big data analytics, machine learning, deep learning, and stochastic optimization. In addition, he is currently visiting professor of data science at the University of Ghent, Belgium. He has published six books, more than 50 research publications, and over 20 patents in the areas of machine learning and information processing. Jimi received his PhD in engineering mathematics from the University of Bristol, U. K., and holds a Bachelor of Science degree from the University of Limerick, Ireland. He is a EU Marie Curie fellow. In 2011 he was selected as a member of the Silicon Valley 50 (Top 50 Irish Americans in Technology).