/tags/wikipedia/ Recent content in Wikipedia on Saleem Ansari Hugo -- gohugo.io en (c) 2024 Saleem Ansari Mon, 03 Feb 2014 00:00:00 +0000 /2014/02/03/a-simple-scala-parser-to-parse-44gb-wikipedia-xml-dump/ Mon, 03 Feb 2014 00:00:00 +0000 /2014/02/03/a-simple-scala-parser-to-parse-44gb-wikipedia-xml-dump/ I had to parse a Wikipedia XML Dump ( 44GB XML file uncompressed ). The XML dump is available here, and I have also created a smaller sample file to run this code: sample wiki.xml file. Below is the XML event based parser using Scala’s XMLEventReader: package xml import scala.io.Source import scala.xml.pull._ import scala.collection.mutable.ArrayBuffer import java.io.File import java.io.FileOutputStream import scala.xml.XML object wikipedia extends App { val xmlFile = args(0) val outputLocation = new File(args(1)) val xml = new XMLEventReader(Source.