There are two types of guilds in ESO.
1. Guilds that are part of the actual game with quests and their own skill lines once joined. These guilds are The Fighters Guild, The Mages Guild and soon The Thieves Guild. There are also The Undaunted but that is more like an organization that focuses on Group Dungeons. 
2. Guilds that are created and owned by other players. You can create one yourself. These guilds does not have their own skill lines or quests but there is alot of other perks.

The guilds joined (except for The Fighets Guild, The Mages Guild, Thieves Guild and The Undaunted) are shared with all your characters on your account. This means that if you joined a guild on your first character and then create a new one, that character will also be a member of that/those guilds. One account can only be a member of up to five guilds (only those created by others is counted into this number).

GUILD BANK
Once you have joined or created a guild you will share a guild bank with all the other members (this function is only activated when a guild has ten or more members). You can then deposit and withdraw items and gold into the bank, but have in mind that these items will be available to all the members so do not use the guild bank for personal storage. Normally it is only the owner and admins that are able to withdraw gold and you will probably not be able to withdraw items at first.

There are ranks within the guild and depending on your dedication to the guild you might be promoted. This will give you more and more perks, like being able to withdraw items from the guild bank.

To access a guild bank you simply go to any bank at your location and then choose "Guild Bank". If you are a member of more than one guild you will have to choose which guild bank you want to access.
There is a maximum of 500 slots in the guild bank.

GUILD STORE
Guild Stores work in two distinct ways.
1. It will allow players in the guild to sell items to one another internally.
2. If the Guild has captured a Keep/Resource in Cyrodiil (more about this under "General Information --> "Campaing/PVP"), there will be an opportunity to sell to guildmates in that keep without having to teleport out of Cyrodiil. 
In Guild Stores, players can set their own prices and sell any item they want, provided it isn't bound to their character or an Imperial Edition item. Since there is no global auction house system in The Elder Scrolls Online, guilds can be created with massive amounts of players for the sole purpose of trading/buying/selling. Guild Tabards are able to be purchased from the Guild Store, regardless of whether or not the guild has more than 50 members, but they will only appear blank until a Guild Heraldry is created (available to guilds with over 10 members). A guild tabard is created by the guild and can be worn by your character over their armor in the "Appearance" slot in your inventory.

GUILD HERALDRY

As I mentioned before there are ranks within the guild.
Guild Heraldry is unlocked once a guild has reached 10 members, and allows a guild to distinguish itself from other guilds. Only the Guildmaster is able to create and modify the Guild's Heraldry. The Guildmaster first must purchase a tabard from the Guild Store, which costs 2000 gold. Then in the new Heraldry tab in the Guild menu, the Guildmaster is able to create the guild's very own tabard. They can select from a very large array of over 250 colors, 63 background styles and 136 crests, as well as a variety of tabard shapes and borders.