December, 1995 LIDS- P 2326Research Supported By:ARO grant DAAL03-92-G-0115ARO grants DAAH04-95-I-0103;Homi Bhabha FellowshipLQG Control with Communication Constraints*Borkar, V.S.Mitter, S.K.December 1995 LIDS-P-2326LQG Control with Communication ConstraintsV. S. Borkar*Department of Computer Science and AutomationIndian Institute of ScienceBangalore 560012IndiaSanjoy IK. MittertDepartment of Electrical Engineering and Computer Scienceand Laboratory for Information and Decision SystemsMassachusetts Institute of Technology35-308Cambridge, Massachusetts 02139U.S.A.December 1995Dedicated to Tom Kailath on the occasion of his sixtieth birthday.AbstractThe average cost control problem for linear stochastic systems withGaussian noise and quadratic cost is considered in the presence ofcommunication constraints. The latter take the form of finite alpha-bet codewords, being transmitted to the controller with ensuing delayand distortion. It is shown that if instead of the state observationsan associated "innovations process" is encoded and transmitted, thenthe separation principle holds, leading to an optimal control linearin state estimate. An associated "off-line" optimization problem forcode length selection is formulated. Some possible extensions are alsopointed out.'This research supported by a Homi Bhabha Fellowship and by the U.S. Army ResearchOffice under grant number DAAL03-92-G-0115tThis research supported by the Army Research Office under grant number DAAL 03-92-G-0115 (Center for Intelligent Control Systems) and grant number DAAH04-95-1-0103(Photonic Networks and Data Fusion).1 INTROD UCTION 2KeywordsLQG control, separation principle, communication constraints, av-erage cost control, optimum code length.1 IntroductionMost traditional analyses of control systems presuppose that the observationvector is available in its entirety to the controller at each decision epoch. Inmany real engineering systems, however, the situation is different. Whatthe controller sees will not often be the original observation vector fromthe sensor, but a quantized version of it transmitted over a communicationchannel with accompanying transmission delays and distortion, subject tobit rate constraints. This calls for control systems analysis that explicitlyaccounts for such communication constraints. This problem has attractedsome attention in recent years, see, e.g. [3, 5, 7, 8, 9]. For related work onmultirate control of sampled-data systems, see [6] and the references therein.The aim of this work is to show that the classical Linear-Quadratic-Gaussian(LQG) problem does admit a rather clean treatment in this framework, withthe proviso that it is not the state or the observation vector that is encodedand transmitted, but an associated process we dub the 'innovations' processby slight abuse of terminology. In fact, a 'separation principle' holds andthis will be the main result of this exercise.There are two key features of our formulation that make this work. Thefirst is the choice of 'innovations process' alluded to above in place of theobservation process as the signal to be quantized and encoded. Unlike thelatter, the former is an i.i.d. Gaussian sequence with statistics independentof control. This allows us to use a fixed optimal vector quantizer for whichextensive analysis is available for the Gaussian case [4]. Secondly, the leastsquares estimation at the output end of the channel can now be based only onthe current channel output and does not have to remember the past outputs,as it ideally should, if the observations were to be encoded directly. Thismakes the estimation scheme at the controller end completely transparent.These observations will become self-evident as we proceed.The second key feature is the centroid property of the optimal vectorquantizer, which allows us to interpret the quantized random variable asthe conditional expectation of the original random variable given an appro-priate sub-a-field. This interpretation nicely fits in with the least squaresestimation scheme we use.2 PRELIMINARIES 3The paper is organized as follows. The next section describes the problemformulation in detail. Section 3 derives the optimal controller. Section 4describes the associated optimal code-length selection problem. Section 5sketches some possible extensions.2 PreliminariesConsider the control systemXk+l = AXk + Buk + vk, k > O, (1)wherei. {Xk} is an Rd-valued 'state' process, X0 prescribed,ii. {Uk} is an Rm-valued control process,iii. A E Rdxd, B e Rdxmiv. {vk} is i.i.d. N(O,Q) noise, that is, normally distributed, zero-meanwith covariance Qv. the following 'nonanticipativity' condition holds: {vj,j > k} is inde-pendent of {xj, uj, vj-l, j < k} for all k > 0.Let G E IRdxd, F E R`mxm be prescribed positive semidefinite matrices.Our control problem is to minimizen-Ilimsup E E [XTGXk + UTFUk]n-+oo n7k=Oover {Uk} as above, subject to the communication mechanism describedbelow. Before getting into the details thereof, we lay down the followingassumptions:Al. The pair (A, B) is controllable.A2. The pair (A, G2) is observable.A3. [IA112 _ Amax(ATA) < 1.2 PRELIMINARIES 4The above control problem is well-posed under (A1)-(A2) [2, pp. 228-229]. (A3) will be used later. We come now to the encoding and communi-cation mechanism.Fix an integer M > 1, the 'code length.' Also let N > 1 be anotherinteger, the 'communication delay' given by N = +(M) for some prescribednondecreasing map k : N -+ N. (Typically, b(n) = [n/r] + 1 where [.]represents integer part and r > 0 is the transmission rate in bits per second.)For k > O, letN-1X(k+l)N = ANXkN + E AN-iBukN+i + Vk+l,i=owhere Vk+l = ((k+l)N fori-1(kN+i = Ai--1VkN+j, 0 < i < N.j=oThen {Vk} are i.i.d. N(O,QN) wherei-1Qi = Ai-j-lQ(AT)i-, 0 < i < N.j=oWe call {Ok} the innovations process by abuse of terminology.At time kN, k > O, start transmitting M-bit encoding of Ok. Thetransmission is complete at time (k + 1)N.Let {al,..., ae} denote the range of the vector quantizer, assumed tosatisfy the usual optimality conditions [4, Section 11.2]. Let {A1,..., Ae}denote the finite partition of Rd generated by the vector quantizer, suchthat Ai gets mapped to ai, 1 < i < e. Let (k denote the a-field generatedby the events {vk E Ai}, i < i < e. Then the centroid condition of optimalvector quantizer [4, p. 352] translates intoVk = E[Vk/(k], k > 0.Letting Pi = P(vk E Ai), 1 < i < e, it is clear thatE[Ok] = piai = 0.We assume a memoryless channel that maps ai