What is the difference between Custom Settings and Custom Metadata Types?

Custom Metadata is a relatively new feature introduced in spring 2015 release. Custom Metadata can do anything that a list custom settings can do.

In short custom metadata types are primarily used to develop peers of Salesforce standard metadata types such as custom fields, tabs, workflows etc.