As grawity said, ~/.bashrc is what you want, since it is sourced by non-interactive non-login shells.
I expect the problem you’re having has to do with the default Ubuntu ~/.bashrc file. It usually starts with something like this:
- If not running interactively, don’t do anything
[ -z “$PS1” ] && return
You want to put anything for non-interactive shells before this line.