Diets fail because they are rigid and fragile.
For a diet to work long term then it must be flexible and anti-fragile. A flexible diet means that there are strategies you have on hand to employ whenever you need to deviate from your plan, that don't ruin things for you. An anti-fragile diet means that your diet gets stronger in the face of volatility, not weaker.
But most of all, a diet has to become part of your lifestyle, a lifestyle you had better enjoy, or yet again, the diet will fail. Your dream lifestyle and a diets success will always be inextricably linked because everything we do in our lives, we do for a particular feeling.
Feel a crappy feeling too often, like hunger, tiredness, low performance or boredom, than, given the opportunity, you will break that feeling up with something that cause pleasure: food.