Updated on: January 29, 2014

Congressional Hearings Focus on End-to-End Testing

By Mark Lott
Original story posted on: October 25, 2013

Finally, the critical importance of ICD-10 testing in healthcare has a national stage. The overwhelming evidence that such testing was overlooked and not performed properly is being highlighted during the current U.S. Department of Health and Human Services (HHS) hearings concerning the launch of the healthcare.gov insurance exchange website.

As professional testers, we see this type of laissez-faire approach to general testing on a daily basis, and the same mistakes are on display in ICD-10 testing as well. Lack of testing always has had the same reasoning behind it: not enough time, not enough money, not enough environments, it’s too hard, etc. Healthcare has never fully invested in testing, which is clearly evident in the complete lack of end-to-end testing environments.

The industry is still learning what other industries have known for decades: that it costs 50 to 100 times more to fix defects once they hit production mode. We have seen every excuse in the book used to justify why testing can’t be done properly, all of which reflect why it doesn’t hold the importance it should.


End-to-end testing is a new concept for healthcare, and the industry is not very well prepared to handle it as of today. In terms of ICD-10, the numbers are not promising, as the industry will be lucky if 2 percent of trading partner relationships are actually tested from end to end. That 2 percent represents only the biggest institutions, as the majority of small hospitals and physician groups are being told on a daily basis that they will not be able to be test with health plans. This leaves the majority of the industry in the dark as to how ICD-10 will affect bottom lines. Is the industry saying no to end-to-end testing arbitrarily? No, they are not doing it because they simply do not have the infrastructure, tools, processes and resources to perform it properly.

The Centers for Medicare & Medicaid Services (CMS) has already said that it will not end-to-end test Medicare FFS, and to date we have not heard definitive plans from CMS that either Medicare or Medicaid will be available for end-to-end testing – even though end-to-end testing is in the regulation for health plans to attest that it was performed. 

The healthcare industry still is using outdated waterfall and similar linear testing approaches, which leaves little time for proper testing and doesn’t reflect the reality of how healthcare changes are implemented. Unless iterative testing is implemented on a large scale, healthcare will continue to undertake costly testing efforts with little or no return on investment.

There are too many silos in healthcare, and it continues to bog down testing efforts because organizations do not understand that each one of them is directly impacted by the quality of their trading partners’ systems – and that relationship needs to be tested. End-to-end testing in healthcare today is nothing more than window dressing that more closely resembles interface testing than full, production-scale testing.

The ability to find problems faster by uncovering system and coding issues sooner has to be achieved using more agile methodologies, because we cannot wait for everything to be complete before systems are thoroughly tested. Testing must be incremental and performed often, not in a two-month window at the very end of a project, when there is no time left to fix anything and bad coding goes into use. The equivalence of the agile model in ICD-10 testing is the Lott Method for Asynchronous Testing. This method provides a test plan and a tool for providers and health plans so they can begin testing now, regardless of where they’re at in their systems remediation process. If your systems will not be ready until March 2014, you can’t afford to wait until then to begin testing everything. There will simply not be enough time—as we saw from the health insurance marketplaces. 

About the Author

Mark Lott is CEO of the Lott QA Group and chairman of the HIMSS-WEDI ICD-10 Testing Workgroup. He also serves as co-chairman of the NCHICA ICD-10 Testing Pilot and is a  member of the ICD-10monitor editorial board.

Contact the Author

To comment on this article please go to

Disclaimer: Every reasonable effort was made to ensure the accuracy of this information at the time it was published. However, due to the nature of industry changes over time we cannot guarantee its validity after the year it was published.