An FTP Client is a software designed to transfer files back-and-forth between two computers over the Internet. It needs to be installed on your computer and can only be used with a live connection to the Internet.
With an FTP Client (such as FileZilla, CuteFTP or SmartFTP) uploading/downloading a file is really easy. Furthermore, FTP clients have some extra features. For example, you're can resume a download that did not finish successfully. This is a very nice feature for people with slower Internet connections.
FTP clients' interface is usually divided in two - the panel on the left shows the files and folders on your computer (your Desktop, My Documents, etc,) and the pane on the right displays the files on the remote server (if this is your website hosting account, you will see the public_html folder, where the files of your site are stored).