Here are some free software solutions for publishing-related tasks. These are programs that are "mature," meaning well supported and very functional.
IMAGE MANIPULATION
The GIMP: This can do most of what Photoshop can do, especially if you take advantage of the various plugins available.
Paint.NET: Another replacement for Photoshop, and is supposed to be very good.
Inkscape: This is a wonderful program for working with lines, text, vector art, etc. It has totally replaced Illustrator for me.
DESKTOP PUBLISHING
LibreOffice: This is an excellent replacement for MS Word. It is capable of being used as a layout program, though it is not as powerful as other software that is specialized for the task. Having said that, you can get good results for layout.
Scribus: This is a proper layout program, though the learning curve is steep and it has its own complications and limitations. [UPDATE] I've been playing around with this and IMHO the learning curve is not actually that bad. Handling tables is kind of fiddly but otherwise this is a great program.
WEBSITE DESIGN
Amaya: An open source alternative to Dreamweaver.
Weebly can be used to design a simple but clean website to get you started. If you learn to fiddle with HTML you can customize your site based on what you do in Weebly.
Namecheap.com: I use this site for registering domain names.