618 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 15, NO. 2, MAY 2000Optimal Electricity Supply Bidding by MarkovDecision ProcessHaili Song, Chen-Ching Liu, Fellow, IEEE, Jacques Lawarrée, and Robert W. Dahlgren, Member, IEEEAbstract—The bidding decision making problem is studiedfrom a supplier’s viewpoint in a spot market environment. Thedecision-making problem is formulated as a Markov DecisionProcess - a discrete stochastic optimization method All othersuppliers are modeled by their bidding parameters with cor-responding probabilities. A systematic method is developedto calculate transition probabilities and rewards. A simplifiedmarket clearing system is also included in the implementation.A risk-neutral decision-maker is assumed, the optimal strategyis calculated to maximize the expected reward over a planninghorizon. Simulation cases are used to illustrate the proposedmethod.Index Terms—Electricity Market, Bidding Strategies, Decision-making, Markov Decision Process, Power System Economics.I. INTRODUCTIONTHE power industry is evolving into an open-access, com-petitive environment. In this environment, economics andprofitability are primary objectives of the market players. Foreach generation or distribution company, decisions have to bemade on transactions, e.g., contract types and parameters. Elec-tricity and services can be sold or purchased through bilateralcontracts or the spot market [1]. The spot market usually op-erates as a pool, i.e., the market participants submit bids to amarket that determines the transactions based on rules agreedupon by the participants.To achieve effciency in generation and consumption ofelectricity, an economic pricing scheme plays an importantrole. Properties of the prices of optional forward contracts arediscussed in [2]. Alternative policies concerning access to andpricing of transmission are studied in [3]. Game theory modelsare used to estimate the possible effects of various policiesupon productive efficiency and the distribution of gains amongall market players.In this paper, the problem of bidding decision-making isstudied from the viewpoint of a generation company. Strategicbidding behavior has been studied in other fields such ascommodity markets but less so in the electricity market. In [4],competitors are modeled by probability distributions of theirbids. A method for updating probability distributions whenManuscript received March 16, 1998; revisedFebruary3,1999.Thisresearchis sponsored by National Science Foundation under Grant no. ECS-9612636with matching support from ALSTOM ESCA Corporation.H. Song and C.-C. Liu are with the Department of Electrical Engineering,University of Washington, Seattle WA 98195.J. Lawarrée is with the Department of Economics, University of Washington,Seattle WA 98195.R. W. Dahlgren is with the ALSTOM ESCA Corp. Bellevue, WA.Publisher Item Identifier S 0885-8950(00)03796-2.new data are observed is discussed. Some studies on biddingstrategies have been conducted for the electricity market. In[5], a framework for an energy brokerage is proposed and asub-optimal bidding strategy is developed according to thecompetitor’s bidding probability density function. In [6], gametheory is used to determine the suppliers’ pricing strategyand it is assumed that suppliers bid with linear marginal pricefunctions without capacity limits. Game theory is also used in[7] to simulate the decision making process for defining offeredprices in a deregulated environment. A genetic algorithm isdeveloped in [8] to select bidding strategies in the doubleauction electricity marketplace. Technical issues related to anauction and bidding market structure are analyzed in [9].Besides bidding and pricing methods, other studies havebeen conducted on the subject of electricity transactions. Forexample, to handle the increasing number of electric energytransactions, an algorithm is proposed for identification ofconflicting conditions between contracts [10].In the literature, the optimal strategy is one that gives thedecision-maker the maximum expected return for one biddingperiod. However, in the daily electricity market, the decision-maker’s bid may influence the future market or the action mayaffect his/her own market position in the future. For example, asupplier’s bid can affect the spot price and the other suppliers’bidding behavior may change according to the spot price. A sup-plier is also subject to resource constraints. The production limitis obvious for a hydro producer. A gas or coal producer mayhave fuel contracts that define the production limit over a timehorizon. In this environment, the decision-maker should lookinto the future when a bidding decision is to be made. The bid-ding strategy that gives the best profit for one day may not beoptimal when the expected profit over the planning horizon isdesired.The market has various uncertainties, e.g., price and load.Hence, the market model proposed here is stochastic. The de-cision-maker is assumed to be risk-neutral; the optimal strategythat maximizes the expected profit is desirable for the deci-sion-maker. To develop a tractable model, a Markov process isassumed. The purpose of the proposed method is to optimize theexpectedreward overa planning horizon. This paper reports newresults on the application of a Markov Decision Process (MDP)to optimize the bidding decisions. The MDP here is of the dis-crete-state and discrete-time type. MDP provides a systematicway to solve multiple stage probabilistic decision-making prob-lems. In an MDP, the stochastic process evolves in a sequenceof time stages. As shown in Fig. 1, at stage t, the market canbe in any of a number of states numbered from 1 toN. In eachstate, the decision-maker can choose one decision a from a set0885–8950/00$10.00 © 2000 IEEESONG et al.: OPTIMAL ELECTRICITY SUPPLY BIDDING BY MARKOV DECISION PROCESS 619Fig. 1. States and state transition in MDP.of feasible decision options. Corresponding to a decision a, thetransition probability from a state i to another state j is givenby. The decisions from the first stage to the endof the planning horizon form one strategy (policy). The deci-sion-maker receives a rewardfrom each transition. TheMDP model is mathematically well established and its applica-tions can be found in many areas [11]–[12].The significance of the proposed method lies in the fact thatthe method calculates the optimal decision over a planninghorizon. The supplier’s production limit is