Diagnosis and overdiagnosis

What is over-diagnosis? My non-medical opinion is that over-diagnosis happens when a thingy is found that isn't a significant thingy and could have been left alone. A more medical definition and discussion can be found in this study which looked at a bunch of other studies for signs of bias and overdiagnosis. (Yes a study about studies.)

"Overdiagnosis, the detection and diagnosis of a condition that would not go on to cause symptoms or death in the patient’s lifetime, is an inevitable harm of screening. Overdiagnosis in cancer screening can result from non-progression of the tumor or from competing mortality due to other patient conditions (that is, other conditions that would lead to the patient’s death before the cancer would have caused symptoms). The consequences of overdiagnosis include unnecessary labeling of people with a lifelong diagnosis as well as unneeded treatments and surveillance that cause physical and psychosocial harm. A patient who is overdiagnosed cannot benefit from the diagnosis or treatment but can only be harmed."

Re-read that last line. 'A patient who is overdiagnosed cannot benefit from the diagnosis or treatment but can only be harmed.' Isn't there a line in the Hippocratic oath saying 'do no harm'? This is the root of the problem.

Medical technology has advanced to the point where it detects small anomalies which are often too quickly diagnosed as 'bad things' when they could just be weird thingies that can be left alone. All these screenings we have contribute to overdiagnosis. How many people have had polyps found in colonoscopies which are nothing but force the patient into more frequent tests? Or a woman who is diagnosed with DCIS - will it ever become symptomatic? We don't know. Or do you know anyone who, after going through a cancer diagnosis and treatment, was later determined to be at a lower stage and didn't need as much treatment as they had had?

To me, this is the crux of the problem. We see too quick to say 'its cancer, get rid of it' when we don't always know. We have lots of tests to find cancer but do we have enough tests to determine if a cancer will be symptomatic? Is enough done before treatment starts to learn as much about the cancer to make sure the treatment matches the correct diagnosis?

I strongly feel this should be an area of significant research. We know so much about cancer now and spend a billions each year to treat cancer and find a cure. Shouldn't we back up and do more research on trying to distinguish which cancers are going to progress and which are not?