The important thing to understand is the convergence of the series for $\exp(t)$.
We'd like to find out if the sum$$\sum_{n=0}^\infty \frac{t^n}{n!}$$is defined for all real $t$ (I'm just going to focus on reals here).
Dealing with infinite sums can be tricky. If you're unsure about whether a particular series converges, and start manipulating it as if it does, you can get weird (and sometimes wrong) answers.
Finite sums, on the other hand, are much safer to deal with. So one approach for questions about convergence is to consider the sequence of "partial sums": in this case, define$$S_N(t)=\sum_{n=0}^N \frac{t^n}{n!}$$
This is defined for all real $t$ and for all non-negative whole numbers $N$ - that is, given a $t$ value and an $N$ value we could (in theory) work out this sum. (The motivation for doing this is that it's often a lot easier to work with sequences than series.)
Let's start with positive $t$. The sequence $S_N$ is certainly increasing (it's formed by adding up positive numbers). If we can show it's bounded above, then we have proved it converges.
Now, for any positive real $t$, let $T$ be the largest integer less than or equal to $t$. Then for any $n>T$,$$\frac{t^n}{n!}=\frac{t^T \cdot t^{n-T}}{T!\cdot (T+1)\cdots n} \le \frac{t^T}{T!}\cdot \left(\frac{t}{T+1}\right)^{n-T}$$
But the number $u=\frac{t}{T+1}$ is positive and less than $1$ (by definition). So for any $N>T$ we have$$S_N(t) \le S_T(t)+\frac{t^T}{T!}\sum_{n=T+1}^N p^{n-T}=S_T(t)+\frac{t^T}{T!}\sum_{k=1}^{N-T} p^k$$
But now we have a finite number ($S_T(t)$) and the sum of a geometric series with a positive ratio smaller than $1$. This proves that the sequence $S_N(t)$ is bounded above for any positive $t$, so it must converge.
OK. This proof ran a bit long because of the commentary. The next step is to consider negative $t$; perhaps you can have a go at that? (A similar approach works: define the partial sums as before, and prove they converge. This time the sequence of partial sums is not strictly increasing, since every other term is negative, so you might need a subsequence).