If No One Sees What the Team is Building...
A friend of mine was telling me about a recent "agile failure" at his organization:
A team has been working on a new product, and the organization has given them everything they ask for--the team has been able to design its physical space and buy new furniture for maximum collaboration. The organization trusts the team, its Scrum Master, and its Product Owner immensely. But after a year it is suddenly discovered that the team has not delivered a valuable product. Where did agile go wrong?
While that short summary clearly leaves out a lot of details, the team was allegedly using scrum, which means the team should've been having a sprint review at the end of each iteration--where were the stakeholders? I love it when an organization trusts its scrum teams, but I worry when the organization's stakeholders aren't engaged and giving feedback. A scrum team is not an island. It exists within an organization, and it needs input from various parts of the organization in order to be truly successful.
Burndown charts and other reports don't provide the same opportunities to collaborate with a scrum team to deliver a more valuable product as an effective sprint review meeting--particularly if the reports are falsified, as they were in this example. If an organization is spending money on a team, it should be interested in seeing the results of their investment, and I suggest looking at the working software that is produced. I don't know of any metrics or KPIs that are going to help an organization prevent a team from lying to them, but I'd suggest asking how many people have attended a sprint review and seen the working software.