{"type":"ScholarlyArticle","title":"An Intelligent Broker Agent for Energy Trading: An MDP Approach","authors":[{"name":"Rodrigue Talla Kuate","slug":"rodrigue-talla-kuate","given":"Rodrigue Talla","family":"Kuate","affiliation":"School of Engineering and Applied Sciences , Aston University Birmingham , United Kingdom"},{"name":"Minghua He","slug":"minghua-he","given":"Minghua","family":"He","affiliation":"School of Engineering and Applied Sciences , Aston University Birmingham , United Kingdom"},{"name":"Maria Chli","slug":"maria-chli","given":"Maria","family":"Chli","affiliation":"School of Engineering and Applied Sciences , Aston University Birmingham , United Kingdom"},{"name":"Hai H. Wang","slug":"hai-h-wang","given":"Hai H.","family":"Wang","affiliation":"School of Engineering and Applied Sciences , Aston University Birmingham , United Kingdom"}],"abstract":"This paper details the development and evaluation of AstonTAC, an energy broker that successfully participated in the 2012 Power Trading Agent Competition (Power TAC). AstonTAC buys electrical energy from the wholesale market and sells it in the retail market. The main focus of the paper is on the broker's bidding strategy in the wholesale market. In particular, it employs Markov Decision Processes (MDP) to purchase energy at low prices in a day-ahead power wholesale market, and keeps energy supply and demand balanced. Moreover, we explain how the agent uses Non-Homogeneous Hidden Markov Model (NHHMM) to forecast energy demand and price. An evaluation and analysis of the 2012 Power TAC finals show that AstonTAC is the only agent that can buy energy at low price in the wholesale market and keep energy imbalance low.","doi":null,"url":"https://ijcai.org/proceedings/2013/44","pdf":"https://ijcai.org/proceedings/2013/0044.pdf","markdown":"https://ijcai.org/proceedings/2013/44.md","bibtex":"https://ijcai.org/proceedings/2013/bibtex/44","volume":{"slug":"2013","title":"Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2013","url":"https://ijcai.org/proceedings/2013/","year":2013,"location":"Beijing, China","dates":"3–9 August 2013","editors":"Francesca Rossi","isbn":"978-1-57735-633-2","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":44,"pages":{"first":234,"last":239},"keywords":[],"videos":[],"references":[{"position":1,"raw":"[Bach et al., 2012] T. Bach, J. Yao, J. Wang, and S. Yang. Research and application of the Q-learning for wholesale power markets. In 2nd International Conference on Con- sumer Electronics, Communications and Networks, pages 1192-1197. IEEE, 2012.","title":"Research and application of the Q-learning for wholesale power markets","authors":"T Bach, J Yao, J Wang, S Yang","year":"2012","venue":"2nd International Conference on Consumer Electronics, Communications and Networks","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio and Frasconi, 1995] Y. Bengio and P. Frasconi. An input output HMM architecture. In G. Tesauro, D. Touret- zky, and T. Leen, editors, Advances in neural information processing systems, volume 7, pages 427-434. MIT Press, Cambridge, MA, 1995.","title":"An input output HMM architecture","authors":"Y Bengio, P Frasconi","year":"1995","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bishop, 2006] C.M. Bishop. Pattern Recognition and Ma- chine Learning. Springer Science and Business Media, LLC, 2006.","title":"Pattern Recognition and Machine Learning","authors":"C M Bishop","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cancelo et al., 2008] J.R. Cancelo, A. Espasa, and R. Grafe. Forecasting the electricity load from one day to one week ahead for the spanish system operator. International Jour- nal of Forecasting, 24(4):588-602, 2008.","title":"Forecasting the electricity load from one day to one week ahead for the spanish system operator","authors":"J R Cancelo, A Espasa, R Grafe","year":"2008","venue":"International Journal of Forecasting","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Conejo et al., 2005] A.J. Conejo, M.A. Plazas, R. Espinola, and A.B. Molina. Day-ahead electricity price forecasting using the wavelet transform and ARIMA models. IEEE Transactions on Power Systems, 20(2):1035-1042, 2005.","title":"Day-ahead electricity price forecasting using the wavelet transform and ARIMA models","authors":"A J Conejo, M A Plazas, R Espinola, A B Molina","year":"2005","venue":"IEEE Transactions on Power Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Contreras et al., 2003] J. Contreras, R. Espinola, F.J. No- gales, and A.J. Conejo. ARIMA models to predict next- day electricity prices. IEEE Transactions on Power Sys- tems, 18(3):1014-1020, 2003.","title":"ARIMA models to predict nextday electricity prices","authors":"J Contreras, R Espinola, F J Nogales, A J Conejo","year":"2003","venue":"IEEE Transactions on Power Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gao et al., 2000] F. Gao, X. Guan, X.R. Cao, and A. Pa- palexopoulos. Forecasting power market clearing price and quantity using a neural network method. In Power Engineering Society Summer Meeting, volume 4, pages 2183-2188. IEEE, 2000.","title":"Forecasting power market clearing price and quantity using a neural network method","authors":"F Gao, X Guan, X R Cao, A","year":"2000","venue":"Power Engineering Society Summer Meeting","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Garcia et al., 2005] R.C. Garcia, J. Contreras, M. Van Akkeren, and J.B.C. Garcia. A GARCH forecasting model to predict day-ahead electricity prices. IEEE Transactions on Power Systems, 20(2):867-874, 2005.","title":"A GARCH forecasting model to predict day-ahead electricity prices","authors":"R C Garcia, J Contreras, M Van Akkeren, J B C Garcia","year":"2005","venue":"IEEE Transactions on Power Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Harvey and Koopman, 1993] A. Harvey and S.J. Koopman. Forecasting hourly electricity demand using time-varying splines. Journal of the American Statistical Association, 88(424):1228-1236, 1993.","title":"Forecasting hourly electricity demand using time-varying splines","authors":"A Harvey, S J Koopman","year":"1993","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ketter et al., 2012] W. Ketter, J. Collins, P. Reddy, C. Flath, and M. Weerdt. The Power Trading Agent Competition. ERIM Report Series, 2012.","title":"The Power Trading Agent Competition","authors":"W Ketter, J Collins, P Reddy, C Flath, M Weerdt","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li and Littman, 2005] L. Li and M.L. Littman. Lazy ap- proximation for solving continuous finite-horizon MDPs. In Proceedings of the National Conference on Artificial In- telligence, volume 20, page 1175. Menlo Park, CA; Cam- bridge, MA; London; AAAI Press; MIT Press, 2005.","title":"Lazy approximation for solving continuous finite-horizon MDPs","authors":"L Li, M L Littman","year":"1175","venue":"Proceedings of the National Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Littman, 1996] M.L. Littman. Algorithms for sequential de- cision making. PhD thesis, Brown University, 1996.","title":"Algorithms for sequential decision making","authors":"M L Littman","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mandal et al., 2005] P. Mandal, T. Senjyu, K. Uezato, and T. Funabashi. Several-hours-ahead electricity price and load forecasting using neural networks. In Power Engi- neering Society General Meeting, volume 3, pages 2146- 2153. IEEE, 2005.","title":"Several-hours-ahead electricity price and load forecasting using neural networks","authors":"P Mandal, T Senjyu, K Uezato, T Funabashi","year":"2005","venue":"Power Engineering Society General Meeting","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nguyen and Nabney, 2010] H.T. Nguyen and I.T. Nabney. Short-term electricity demand and gas price forecasts us- ing wavelet transforms and adaptive models. Energy, 35(9):3674-3685, 2010.","title":"Short-term electricity demand and gas price forecasts using wavelet transforms and adaptive models","authors":"H T Nguyen, I T Nabney","year":"2010","venue":"Energy","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Power TAC Community, 2013a] Power TAC Community. Overview for tournament: finals-2012-12. http://wolf- 08.fbk.eur.nl:8080/TournamentScheduler/faces/tourna- ment.xhtml?tournamentId=9, 2013. [Online; accessed 28-January-2013].","title":"Power TAC Community. Overview for tournament: finals-2012-12","authors":null,"year":"2013","venue":null,"doi":null,"url":"http://wolf-08.fbk.eur.nl:8080/TournamentScheduler/faces/tourna-ment.xhtml?tournamentId=9,2013","internal_url":null},{"position":16,"raw":"[TAC Community, 2013b] Power TAC Com- munity. Power Trading Agent Competition. http://www.powertac.org, 2013. [Online; accessed 28-January-2013].","title":"Power TAC Community. Power Trading Agent Competition","authors":null,"year":"2013","venue":null,"doi":null,"url":"http://www.powertac.org,2013","internal_url":null},{"position":17,"raw":"[Ramanathan et al., 1997] R. Ramanathan, R. Engle, C.W.J. Granger, F. Vahid-Araghi, and C. Brace. Short-run fore- casts of electricity loads and peaks. International Journal of Forecasting, 13(2):161-174, 1997.","title":"Short-run forecasts of electricity loads and peaks","authors":"R Ramanathan, R Engle, C W J Granger, F Vahid-Araghi, C Brace","year":"1997","venue":"International Journal of Forecasting","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Singh, 1993] S.P. Singh. Learning to solve Markovian deci- sion processes. PhD thesis, University of Massachusetts, 1993.","title":"Learning to solve Markovian decision processes","authors":"S P Singh","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Song et al., 2000] H. Song, C.-C. Liu, J. Lawarrée, and R.W. Dahlgren. Optimal electricity supply bidding by Markov Decision Process. IEEE Trans. Power Systems, 15(2):618-624, 2000.","title":"Optimal electricity supply bidding by Markov Decision Process","authors":"H Song, C.-C Liu, J Lawarrée, R W Dahlgren","year":"2000","venue":"IEEE Trans. Power Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sutton and Barton, 2000] R.S. Sutton and A.G. Barton. Re- inforcement Learning, an Introduction. The MIT press, 2000.","title":"Reinforcement Learning, an Introduction","authors":"R S Sutton, A G Barton","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[TAC Community, 2013] TAC Community. Trading Agent Competition. http://www.sics.se/tac, 2013. [Online; ac- cessed 28-January-2013].","title":"TAC Community. Trading Agent Competition","authors":null,"year":"2013","venue":null,"doi":null,"url":"ac-cessed28-","internal_url":null},{"position":22,"raw":"[Tellidou and Bakirtzis, 2006] A.C. Tellidou and A.G. Bakirtzis. Multi-agent reinforcement learning for strategic bidding in power markets. In 3rd International Conference on Intelligent Systems, pages 408-413. IEEE, 2006.","title":"Multi-agent reinforcement learning for strategic bidding in power markets","authors":"A C Tellidou, A G Bakirtzis","year":"2006","venue":"3rd International Conference on Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Watkins, 1989] C.J. Watkins. Learning from Delayed Re- wards. PhD thesis, Cambridge University, 1989.","title":"Learning from Delayed Rewards","authors":"C J Watkins","year":"1989","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yao et al., 2000] S.J. Yao, Y.H. Song, L.Z. Zhang, and X.Y. Cheng. Wavelet transform and neural networks for short- term electrical load forecasting. Energy Conversion and Management, 41(18):1975-1988, 2000.","title":"Wavelet transform and neural networks for shortterm electrical load forecasting","authors":"S J Yao, Y H Song, L Z Zhang, X Y Cheng","year":"2000","venue":"Energy Conversion and Management","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang and Luh, 2005] L. Zhang and P.B. Luh. Neural network-based market clearing price prediction and con- fidence interval estimation with an improved extended Kalman filter method. IEEE Transactions on Power Sys- tems, 20(1):59-66, 2005.","title":"Neural network-based market clearing price prediction and confidence interval estimation with an improved extended Kalman filter method","authors":"Luh Zhang, ] L Zhang, P B Luh","year":"2005","venue":"IEEE Transactions on Power Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zheng et al., 2005] H. Zheng, L. Xie, and L.-Z. Zhang. Electricity price forecasting based on GARCH model in deregulated market. In The 7th International Power Engi- neering Conference, pages 1-410. IEEE, 2005.","title":"Electricity price forecasting based on GARCH model in deregulated market","authors":"H Zheng, L Xie, L.-Z Zhang","year":"2005","venue":"The 7th International Power Engineering Conference","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}