Configuring ssh for Jenkins for Git on Ubuntu

I ran into some problems configuring ssh for my jenkins box. It turns out that when installing jenkins on ubuntu via the package manager that jenkins is setup to run as its own user. So, Jenkins did not have access to my .ssh directory. Of course this made perfect sense once I stopped and thought about it.