You can change the Guild name and avatar by clicking on the edit profile button
You can change the Guild description by clicking on the Edit description button.
Remember that this is the Guild "business card". This is what players looking for a Guild will see so make sure that what you write there is interesting if you want to attract new members
See Guild rights for more details