I have started working on this problem since 2003 yet a mathematical filter was developed since 2010 and finalized in 2013.
The filter solves the noise from data from individual molecules: e.g. biopolymers (proteins, DNA, RNA, etc), nano – crystals, ion channels: that is the filter removes the noise resulting in clean data. We show that the filter working also in cases where binning fails.
The filter is a numerical algorithm with various new statistical treatments. It is based on a new general likelihood function developed here, with observable dependent form. The filter can solve the noise, in the detectable region of the rate space: that is, we also find a region where the data is “too†noisy. Consistency tests will find the region’s type from the data. If the data is ruled “too noisyâ€, binning obviously fails, and one should apply simpler methods on the raw data and realizing that the extracted information is partial. We show that not applying the filter while cleaning results in erroneous rates. This filter (with minor adjustments) can solve the noise in any discrete state trajectories, yet extensions are needed in “tackling†the noise from other data, e.g. continuous data and FRET data.
The filter developed here is complementary with our previous projects in this field, where we have solved clean two state data with the development of reduced dimensions forms (RDFs): unique models that are canonical forms of two state data, and with the development of a statistical and numerical toolbox that builds a RDF from finite, clean, two – state data. Thus, only the combined procedures enabling building the most accurate model from noisy trajectories from single molecules
The project was submitted during Spring 2013