e is the base for which the exponential function is its own derivative, and that is the whole of it.
For any base a, the derivative of a to the power x is a to the power x, multiplied by some constant that depends on a. For a = 2 that constant is about 0.693, for a = 3 about 1.099. Somewhere between them there is a base where the constant is exactly 1.
That base is e. It is not chosen for being 2.718; 2.718 is what you measure once you insist on the constant being 1.
Every other appearance follows from that. Compound interest, the normal distribution, radioactive decay: all of them involve something whose rate of change is proportional to its size, and e is the number built for exactly that.