An RSS feed is nothing more than a normal XML file. It has to follow ISO standards, but they are very easy to set up. Looking something like..
CODE
<?xml version="1.0" ?>
<rss version="2.0">
<channel>
<title>Kuato's Website</title>
<description>Learn all about Kuato's exciting life!</description>
<link>http://myserver.com</link>
<item>
<title> -- story title -- </title>
<description> -- story description -- </description>
<link>http://myserver.com/story.php?id=1</link>
</item>
</channel>
</rss>
You could copy and paste that text above, save it to a text file, rename it to have an .rss extension, upload it to your site then point an RSS reader at it and it will work. To add more entries just keep repeating the <item> ... </item> blocks.
Now you have an RSS feed you need to work out whether you are wanting to update the file manually or automatically. If it's the latter then I would need to know more about the hosting package you have, the server OS type, what server side languages are available to you and if you use a database to hold the story details.