Admin --> Categories --> Click on the Catogory which you want to delete
Then select (Check Box) Delete this category --> Save Changes
If you have sub-categories this option does not show for parent categories. First you have to delete sub-catogeries.
To delete sub-categories click on sub-categories which are shown under parent catogery.