Scientific Applications of Machine LearningScientific Imagery ApplicationsSome Basic Machine Learning DistinctionsCorrespondence ProblemsMixture ModelsStochastic Grammars for Data ModelingText & Biology ModelsMore Detailed Clustering GrammarsRock Field GrammarTranscriptional Gene Regulation NetworksGene Regulation + Signal Transduction NetworkSoftware architectures for systems biology: Sigmoid & Cellerator3-tier architecturePossible software supportMetadata in Systems BiologyPowerPoint PresentationSAM gene network: ResultsSAM: Gene Network ModelSAM growth imagery PIN1 cell wallsSlide 20Basic Machine Learning DistinctionsContactsUCI ICS IGB SISLScientific Applications of Machine LearningEric MjolsnessScientific Inference Systems LaboratoryDonald Bren School of Information and Computer Sciences, and Institute for Genomics and BioinformaticsUniversity of California, IrvineUCI ICS IGB SISLScientific Imagery ApplicationsNGC 7331 - http://photojournal.jpl.nasa.gov/catalog/PIA06322Arabidopsis SAM - Meyerowitz LabUCI ICS IGB SISLSome Basic Machine Learning Distinctions•Supervised vs. unsupervised learning–Supervised e.g. classification and regression •Feature selection•regression for phenomenological model fitting e.g. GRN’s–Unsupervised e.g. clustering; may be preprocessor•Generative vs. Kernal methods–Generative (statistical inference) models–Kernal methods e.g Support Vector Machines•Vector vs. Relationship data–Vector data: preprocessed image features log I, x, …–Images, time series, shifted spectra - semigroup actions–Sparse graph/relationship data - permutation actionsUCI ICS IGB SISLCorrespondence Problems•Extended sources - map morphologies–Similar to biological imaging problems–Fewer sources but many pixels•Moving or changing point sources–E.g. Ida and Dactyl / JPL MLS•Dense point sources with instrument noise e.g. globular clusters (radial density function)•Techniques: –soft permutations, geometric transformations via optimization & continuation–Embedding inside a graph clustering (optimization) algorithm–Multiscale acceleration of optimizationUCI ICS IGB SISLMixture Models•Mixture of Gaussians, t-distributions, …–Can do outlier detection•Mixture of factor analyzers•Mixture of time series models•* Problem-specific generative models–Can formulate with a Stochastic Parameterized Grammar–Clustering graphsFrey et al. 1998Utsugi and Kumagai 2000UCI ICS IGB SISLStochastic Grammars for Data ModelingUCI ICS IGB SISLText & Biology ModelsUCI ICS IGB SISLMore Detailed Clustering Grammars•Clusters generate data•Priors on cluster centers & variances•Iterative through levels in a hierarchy•Recursive through hierarchyσPr r y Pr r x −r y ( )/σPrDatasetCluster(1,y,σ)Cluster(2,y,σ)Cluster(3,y,σ)Datu ′ m (1,1),x(1,1)( )Datu ′ m (1,2),x(1,2)( )Datu ′ m (3,n),x(3,n)( )Datumx1( )DatumxN( )Datumx2( )1/N!UCI ICS IGB SISLRock Field Grammargrammar rockfield(){ start → {deposit(a,ya,ca,pa)|a ∈A}, distractors ca2/ 2σ02a∑+ ya2/ 2σ02a∑+ pa−ˆp2/ 2σ12a∑+log2ˆσa/)σ( ) deposit(a,ca,pa) → {patch(a,b,xab,ca,pab)|a∈A,b∈Ba} patch(a,b,xab,ca,pa) → {rock xabc,cabc,sabc( )|a∈A,b∈Ba,c ∈Cab} yab−yaa∑2/ 2ˆσa2 pab~ f pa, xab−xa( ) cabc−cabab∑2/ 2σ42+ yabc−yabab∑2/ 2σ52 ⇒MFTE = ca2/ 2σ02a∑+ ya2/ 2σ02a∑+ pa−ˆp2/ 2σ12a∑+log2ˆσa/)σ( ) + yab−yaa∑2/ 2ˆσa2+ Piabci−ca2/ 2σ42+ xi−yab2/ 2σ52⎡⎣⎢⎤⎦⎥iab∑ {rock xabc,cabc,sabc( )|a∈′A ,b∈′Ba,c ∈Cab}→ {visible_rock xi,ci,si( )|i ∈I} distractors → {rock x00d,c00d,s00d( )|d ∈D} c00dab∑2/ 2σ02+ x00dab∑2/ 2σ32 s00d~ sizedistrˆp( ){},,~uniform_permutations()iabciabcabicPPn=∑ xi= Pi,abcxabcab∑ sabc~ sizedistr pa( )UCI ICS IGB SISLTranscriptional Gene Regulation Networks•Gene Regulation Network (GRN) modelTvExtracellularcommunicationDrosophila eve stripe expression in model (right) and data (left). Green: eve expression, red: kni expression. From [Reinitz and Sharp, Mech. of Devel., 49:133-158, 1995 ]. [Mjolsness et al. J. Theor. Biol. 152: 429-453, 1991]UCI ICS IGB SISLGene Regulation + Signal Transduction Networktranscriptional regulation targetsreceptorsligandscellnucleus ddtva(t) =1τag(ua+ha)−λava⎡⎣⎤⎦,whereua(t) = Tabvb(t)+b∑ΛIˆTabvbI(t)b∑I ∈Nbrs∑+ ΛI%Tac1%Tcb2vc(t)vbIc∑b∑I ∈Nbrs∑(t)TUCI ICS IGB SISLSoftware architectures for systems biology: Sigmoid & CelleratorUCI ICS IGB SISL3-tier architectureDatabase AccessModel TranslationSigmoidPathway Representation/StorageDatabaseCelleratorSimulation/InferenceEngineMENUInteractiveGraphicModel(SVG/Applet)Graphic OutputOJBAPIJLinkAPIPROPERTYSOAP: Web ServiceXML(Object), Image,via HTTPUCI ICS IGB SISLPossible software support•Machine learning (open source/academic)–CompClust (CIT/JPL): •Scripting/GUI dichotomy data point; •dataset views–WEKA data mining–Intel: PNL Probabilistic Networks Library–Future: stochastic grammar modeler •+ autogeneration (as in Cellerator)•Image processing, data environments –Matlab, IDL, Mathematica, Khoros/VisiQuest, …–NIHImage/ImageJ, …UCI ICS IGB SISLMetadata in Systems Biology•SBML•Sigmoid UMLQuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.UCI ICS IGB SISLCLV3CLV1?Fletcher et al., Science v. 283, 1999Brand et. al., Science 289, 617-619, (2000) WUSUCI ICS IGB SISLprotein concentrations YSAM gene network: Resultswus(init) and L1XUCI ICS IGB SISLSAM: Gene Network ModelZwusclv1clv3XdiffusiveYL1diffusiveUCI ICS IGB SISLSAM growth imageryPIN1 cell wallsQuickTime™ and aTIFF decompressorare needed to see this picture.UCI ICS IGB SISLVenu GonehalUCI ICS IGB SISLBasic Machine Learning Distinctions•Supervised vs. unsupervised learning–Supervised e.g. classification and regression •Feature selection•regression for phenomenological model fitting e.g. GRN’s–Unsupervised e.g. clustering; may be preprocessor•Generative vs. Kernal methods–Generative (statistical inference) models–Kernal methods e.g Support Vector Machines•Vector vs. Relationship data–Vector data: preprocessed image features log I, x, …–Images, time series, shifted spectra - semigroup actions–Sparse graph/relationship data - permutation actionsUCI ICS IGB SISLContacts•Wayne Hayes, UCI ICS faculty–scientific computing•UCI ICS Maching Learning–Padhraic Smyth–Pierre Baldi•Chris