Sorry to hear about the trouble. When Jibo first starts up he will orient away from his power jack. While this might be considered his default front/back orientation, he should be able to adjust and swivel to interact with you from any direction.
If you find Jibo is often looking in the wrong direction in response to your voice, he may be thrown off by the acoustics of his new location. This can happen Jibo is placed too close to walls, corners or in other enclosed spaces. (Cabinets, bookshelves, etc)
If the issue persists, you may need to adjust Jibo somewhat or even choose a different place in your home for him to live. The acoustics of every home are unique and will differ from room to room so this may require some experimentation to find the ideal spot.
You can see some general placement tips for Jibo in this article.
Hope this helps!