Abstract
Parallel programming, whether imperative or functional, has long focused on arrays as the central data type. Meanwhile, typed functional programming has explored a variety of data types, including lists and various forms of trees. Generic functional programming decomposes these data types into a small set of fundamental building blocks: sum, product, composition, and their associated identities. Definitions over these few fundamental type constructions then automatically assemble into algorithms for an infinite variety of data types—some familiar and some new. This paper presents generic functional formulations for two important and well-known classes of parallel algorithms: parallel scan (generalized prefix sum) and fast Fourier transform (FFT). Notably, arrays play no role in these formulations. Consequent benefits include a simpler and more compositional style, much use of common algebraic patterns and freedom from possibility of run-time indexing errors. The functional generic style also clearly reveals deep commonality among what otherwise appear to be quite different algorithms. Instantiating the generic formulations, two well-known algorithms for each of parallel scan and FFT naturally emerge, as well as two possibly new algorithms.
Formats available
You can view the full content in the following formats:
References
[1]
Roland Backhouse, Jeremy Gibbons, Ralf Hinze, and Johan Jeuring. Datatype-Generic Programming: International Spring School, Revised Lectures . Lecture Notes in Computer Science. Springer Berlin Heidelberg, April 2007.
[2]
Jost Berthold, Mischa Dieterle, Oleg Lobachev, and Rita Loogen. Parallel FFT with Eden skeletons . In International Conference on Parallel Computing Technologies, PaCT ’09, pages 73–83, 2009.
[3]
Richard Bird and Lambert Meertens. Nested Datatypes . In Mathematics of Program Construction, pages 52–67, 1998.
[4]
Per Bjesse, Koen Claessen, Mary Sheeran, and Satnam Singh. Lava: Hardware design in Haskell . In International Conference on Functional Programming, pages 174–184, 1998.
[5]
Guy E. Blelloch. Prefix sums and their applications . Technical Report CMU-CS-90-190, School of Computer Science, Carnegie Mellon University, November 1990.
[6]
Guy E. Blelloch. Programming parallel algorithms . Communications of the ACM, 39:85–97, 1996.
[7]
R. P. Brent and H. T. Kung. A regular layout for parallel adders . IEEE Transactions on Computers, 31(3), March 1982.
[8]
James W. Cooley and John W. Tukey. An algorithm for the machine calculation of complex Fourier series. Mathematics of Computation, 19:297–301, 1965.
[9]
Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms. The MIT Press, 3rd edition, 2009.
[10]
G.C. Danielson and C. Lanczos. Some improvements in practical Fourier analysis and their application to X-ray scattering from liquids. Journal of the Franklin Institute, 233(5):435–452, 1942.
[11]
Fer-Jan de Vries. A functional program for the Fast Fourier Transform . SIGPLAN Notices, pages 67–74, January 1988.
[12]
Richard A. Eisenberg, Dimitrios Vytiniotis, Simon L. Peyton Jones, and Stephanie Weirich. Closed type families with overlapping equations . In Principles of Programming Languages, pages 671–684, 2014.
[13]
Richard A. Eisenberg, Stephanie Weirich, and Hamidhasan G. Ahmed. Visible type application . In European Symposium on Programming Languages and Systems, pages 229–254, 2016.
[14]
Conal Elliott. Denotational design with type class morphisms . Technical Report 2009-01, LambdaPix, March 2009.
[15]
Conal Elliott. Compiling to categories . Proc. ACM Program. Lang., 1(ICFP), September 2017.
[16]
Matteo Frigo. A fast Fourier transform compiler . In PLDI, volume 34, pages 169–180. ACM, May 1999.
[17]
Jeremy Gibbons. Upwards and downwards accumulations on trees . In Mathematics of Program Construction, 1993.
[18]
Jeremy Gibbons. Generic downwards accumulations . Science of Computer Programming, 37(1-3):37–65, May 2000.
[19]
Sergei Gorlatch. Programming with divide-and-conquer skeletons: A case study of FFT. The Journal of Supercomputing, 1998.
[20]
Pieter H. Hartel and Willem G. Vree. Arrays in a lazy functional language — a case study: The fast Fourier transform . In 2nd Arrays, functional languages, and parallel systems (ATABLE), 1992.
[21]
Michael T. Heideman, Don H. Johnson, and C. Sidney Burrus. Gauss and the history of the fast Fourier transform. IEEE ASSP Magazine, 1(4):14–21, October 1984.
[22]
Ralf Hinze. Memo functions, polytypically! In 2nd Workshop on Generic Programming, pages 17–32, 2000.
[23]
Ralf Hinze. An algebra of scans . In International Conference on Mathematics of Program Construction, pages 186–210, 2004.
[24]
Gérard Huet. The zipper . Journal of Functional Programming, 7(5):549–554, September 1997.
[25]
John Hughes. Generalising monads to arrows . Science of Computer Programming, 37:67–111, 1998.
[26]
Darius Jahandarie, Conor McBride, and João Cristóvão. newtype-generics, 2014. Haskell library.
[27]
C. Barry Jay. Matrices, monads and the fast Fourier transform . Technical Report UTSSOCS-93.13, University of Technology, Sydney, 1993.
[28]
Steven G. Johnson. Diagram to illustrate the general Cooley-Tukey FFT algorithm, 2010. URL https://en.wikipedia.org/wiki/ Cooley%E2%80%93Tukey_FFT_algorithm#General_factorizations .
[29]
Geraint Jones. Deriving the fast Fourier algorithm by calculation . In Glasgow Workshop on Functional Programming, 1989.
[30]
Geraint Jones. A fast flutter by the Fourier transform. In IV Higher Order Workshop, Banff 1990, pages 77–84, 1991.
[31]
Gabriele Keller and Manuel M. T. Chakravarty. On the distributed implementation of aggregate data structures by program transformation . In Parallel and Distributed Processing, pages 108–122, 1999.
[32]
Gabriele Keller, Manuel M. T. Chakravarty, Roman Leshchinskiy, and Simon Peyton. Regular, shape-polymorphic, parallel arrays in Haskell . In International Conference on Functional Programming, 2010.
[33]
Oleg Kiselyov, Kedar N. Swadi, and Walid Taha. A methodology for generating verified combinatorial circuits . In EMSOFT, 2004.
[34]
Richard E Ladner and Michael J Fischer. Parallel prefix computation. Journal of the ACM, 27(4):831–838, 1980.
[35]
José Pedro Magalhães, Atze Dijkstra, Johan Jeuring, and Andres Löh. A generic deriving mechanism for Haskell . In Haskell Symposium, pages 37–48, 2010.
[36]
José Pedro Magalhães and Andres Löh. A formal comparison of approaches to datatype-generic programming . In Workshop on Mathematically Structured Functional Programming, pages 50–67, March 2012.
[37]
José Pedro Magalhães et al. GHC.Generics, 2011. URL https://wiki.haskell.org/GHC.Generics . Haskell wiki page.
[38]
Conor McBride. The derivative of a regular type is its type of one-hole contexts (extended abstract), 2001. Unpublished.
[39]
Conor McBride and Ross Paterson. Applicative programming with effects . Journal of Functional Programming, 18(1), 2008.
[40]
John T. O’Donnell. A correctness proof of parallel scan . Parallel Processing Letters, 04(03):329–338, 1994.
[41]
David Sharp and Martin Cripps. Synthesis of the fast Fourier transform algorithm by functional language program transformation. In Euromicro Workshop on Parallel and Distributed Processing, pages 136–143, January 1993.
[42]
Mary Sheeran. Parallel prefix network generation: An application of functional programming . In Hardware Design and Functional Languages, 2007.
[43]
Mary Sheeran. Functional and dynamic programming in the design of parallel prefix networks . Journal of Functional Programming, 21(1):59–114, January 2011.
[44]
J. Sklansky. Conditional-sum addition logic. IRE Transactions on Electronic Computers, EC-9(2):226–231, June 1960.
[45]
Brent A Yorgey, Stephanie Weirich, Julien Cretin, Simon Peyton Jones, Dimitrios Vytiniotis, and José Pedro Magalhães. Giving Haskell a promotion . In Workshop on types in language design and implementation, 2012.