Practice question · Multiple choice
A neural network layer is a matrix multiply followed by a non-linear function. Why is the non-linearity essential rather than decorative?
Hints
- Multiply two matrices. Is the result a matrix?
- Ask what a network of pure matrix multiplies could represent that a single matrix could not.
Show the answer
A. Because a composition of linear maps is itself linear
Why
AB is a matrix, so composing linear layers collapses to one, depth without non-linearity is a very expensive way to compute a single linear map. ReLU or similar breaks the collapse, and that is the whole reason deep networks can represent what shallow ones cannot.
Practise Systems of Linear Equations
The app has 5 more questions on this lesson, and keeps your place in the course. Computer Science I is free to start.
More questions on Systems of Linear Equations
- Order the steps of solving 'x + y = 5, 2x + y = 8' by elimination.
- A system of two linear equations in two unknowns can have exactly two solutions.
- Sort each equation by whether it is linear in x and y.
- The system 'x + y = 4, 2x + 2y = 8' has infinitely many solutions. Select every pair (x, y) that is a…