wiki.dbpedia.org - Documentation

2008-07-07 23:12:02

Comparison for /Documentation of 2008-07-07 23:12:02 and 2008-07-07 23:14:29

Additions:

If you want to create your own dumps or work within your IDE or on the console, you should use start.php. If you first want to learn how DBpedia extraction works, or test /debug new extractors, webStart.php might best suit your needs, as it gives you a comfortable web debug interface, similar to http://www4.wiwiss.fu-berlin.de/dbp-debug/. Make sure to download RAP – RDF API for PHP first if you want to use the web interface.

Deletions:

If you want to create your own dumps or work within your IDE or on the console, you should use start.php. If you first want to learn how DBpedia extraction works, or test /debug new extractors, webStart.php might best suit your needs, as it gives you a comfortable web debug interface, similar to http://www4.wiwiss.fu-berlin.de/dbp-debug/. Make sure to download RAP – RdF API for PHP first if you want to use the web interface.

2007-09-04 19:02:10

Comparison for /Documentation of 2007-09-04 19:02:10 and 2008-07-07 23:12:02

Additions:

This pages describes the DBpedia information extraction framework. The framework consists of the interfaces: Destination, Extractor, Page Collection, and RDFnode, plus the essential classes Extraction Group, Extraction Job, Extraction Manager, Extraction Result, and RDFtriple.
The code extraction process is triggered via the ExtractionManager, which starts one or more ExtractionJobs. An ExtractionJob combines one or more ExtractionGroups with a PageCollection. The PageCollection is the data source, e.g., all articles from the Wikipedia SQL-Dump. ExtractionGroups consist of a Destination and one or more Extractors. Possible Destinations include your console, NTriple files and the web interface. Of course you are free to write your own destinations (e.g., databases, RDF/XML files).
Extractors are designed for single specific purposes, e.g., the InfoboxExtractor reads out information from Wikipedia Infoboxes. The ShortAbstractExtractor gets the first paragraph from an article and so on. DBpedia comes with extractors for many purposes already, though you are invited to add your own.

start(): Initializes the Destination (e.g., creates a new NTriple file). Is called once at the beginning of an ExtractionJob.

PageCollections are the data sources for extraction. A PageCollection loads the page source code for a specific language and one or more pages (Implementations: LiveWikipedia, DatabaseWikipedia).
URI, RDFliteral, and RDFblankNode are implementations of RDFnodes.
In addition, information such as datatype, language, and lexical form of

__construct($destination, $metadestination = NULL): $destination is an object of a class, implementing the interface Destination. $metadestination is an optional Destination, where meta information can be stored. It is mainly used by the InfoboxExtractor, which collects all predicate names in a metadestination.

RDFtriple combines a subject, a predicate, and an object, into an RDFtriple. You will usually create RDFnodes via RDFtriples, e.g. $subject = RDFtriple::URI("Resource Name"); as RDFtriple can add a common URI prefix for each triple.

static function literal($value, $datatype = null, $lang = null): Returns an RDFliteral. $value must be of datatype String; else an error message will be thrown, as $value will be serialized for NTriples and this only works with string variables. $datatype and $lang are optional parameters. $datatype is a string containing the (RDF-) datatype of $value, $lang the language.

Deletions:

This pages describes the DBpedia information extraction framework. The framework consists of the interfaces: Destination, Extractor, Page Collection and RDFnode, plus the essential classes Extraction Group, Extraction Job, Extraction Manager, Extraction Result and RDFtriple.
The code extraction process is triggered via the ExtractionManager, which starts one or more ExtractionJobs. An ExtractionJob combines one or more ExtractionGroups with a PageCollection. The PageCollection is the data source, e.g. all articles from the Wikipedia SQL-Dump. ExtractionGroups consist of a Destination and one or more Extractors. Possible Destinations include your console, NTriple files and the web interface. Of course you are free to write your own destinations (e.g. databases, RDF/XML files).
Extractors are designed for single specific purposes. E.g. the InfoboxExtractor reads out information from Wikipedia Infoboxes. The ShortAbstractExtractor gets the first paragraph from an article and so on. DBpedia comes with extractors for many purposes already, though you are invited to add your own.

start(): Initializes the Destination (e.g. creates a new NTriple file). Is called once at the beginning of an ExtractionJob.

PageCollections are the data source for extraction. A PageCollection loads the page source code for a specific language and one or more pages (Implementations: LiveWikipedia, DatabaseWikipedia).
URI, RDFliteral and RDFblankNode are implementations of RDFnodes.
In addition informations such as datatype, language and lexical form of

__construct($destination, $metadestination = NULL): $destination is an object of a class, implementing the interface Destination. $metadestination is an optional Destination, where meta information can be stored. It is mainly used by the InfoboxExtractor, which collects all predicate names in a metdestination.

RDFtriple combines a subject, a predicate and an object to a RDFtriple. You will usually create RDFnodes via RDFtriples. E.g. $subject = RDFtriple::URI("Resource Name"); as RDFtriple can add a common URI prefix for each triple.

static function literal($value, $datatype = null, $lang = null): Returns a RDFliteral. $value must be of datatype String, else an error message will be thrown, as $value will be serialized for NTriples and this only works with string variables. $datatype and $lang are optional parameters. $datatype is a string containing the (RDF-) datatype of $value, $lang the language.

2007-08-23 11:21:49

Comparison for /Documentation of 2007-08-23 11:21:49 and 2007-09-04 19:02:10

Additions:

The DBpedia Information Extraction Framework

The DBpedia community uses a flexible and extensible framework to extract different kinds of structured information from Wikipedia.
The DBpedia information extraction framework is written using PHP 5. The framework is available from the DBpedia SVN (GNU GPL License).
This pages describes the DBpedia information extraction framework. The framework consists of the interfaces: Destination, Extractor, Page Collection and RDFnode, plus the essential classes Extraction Group, Extraction Job, Extraction Manager, Extraction Result and RDFtriple.

Deletions:

Documentation of the DBpedia Extraction Framework

Preliminary Version: This document is not complete and might contain errors.
This pages describes the DBpedia extraction framework. DBpedia data extraction is written entirely in PHP 5. The framework consists of the interfaces Destination, Extractor, Page Collection and RDFnode, plus the essential classes Extraction Group, Extraction Job, Extraction Manager, Extraction Result and RDFtriple.

2007-08-23 11:19:35

Comparison for /Documentation of 2007-08-23 11:19:35 and 2007-08-23 11:21:49

Additions:

static function blank($label): Returns a RDFblankNode. $label must be string containing the Blanknode label.
static function URI($uri): Returns an URI. $uri is a string containing the URI.

2007-08-21 23:31:26

Comparison for /Documentation of 2007-08-21 23:31:26 and 2007-08-23 11:19:35

Additions:

static function literal($value, $datatype = null, $lang = null): Returns a RDFliteral. $value must be of datatype String, else an error message will be thrown, as $value will be serialized for NTriples and this only works with string variables. $datatype and $lang are optional parameters. $datatype is a string containing the (RDF-) datatype of $value, $lang the language.

2007-08-21 23:29:24

Comparison for /Documentation of 2007-08-21 23:29:24 and 2007-08-21 23:31:26

Additions:

Interface Destination

Interface Extractor

Interface PageCollection

Interface RDFnode

Class ExtractionGroup

Class ExtractionJob

Class ExtractionManager

Class ExtractionResult

Class RDFtriple

Deletions:

Interface Destination
Interface Extractor
Interface PageCollection:
Interface RDFnode
Class ExtractionGroup
Class ExtractionJob
Class ExtractionManager
Class ExtractionResult
Class RDFtriple

2007-08-21 23:25:19

Comparison for /Documentation of 2007-08-21 23:25:19 and 2007-08-21 23:29:24

Additions:

To get the framework running on your local PC, it is recommended to start with the pre-configured files start.php or webStart.php. You will find these in the DBpedia root folder. The extraction code is available via Sourceforge SVN (do not download the release version at Sourceforge).
If you want to create your own dumps or work within your IDE or on the console, you should use start.php. If you first want to learn how DBpedia extraction works, or test /debug new extractors, webStart.php might best suit your needs, as it gives you a comfortable web debug interface, similar to http://www4.wiwiss.fu-berlin.de/dbp-debug/. Make sure to download RAP – RdF API for PHP first if you want to use the web interface.
If you intend to produce separate output files for each Extractor, you will need an own ExtractionGroup for each extractor (as is done in the standard settings in start.php). Finally you will need to run your ExtractionJob through an instance of an ExtractionManager.

extractPage($pageID, $pageTitle, $pageSource): Includes the actual extraction process. Constructs a new ExtractionResult, extracts data from the source page and stores the extracted data in the ExtractionResult. Is called for each page and must return an ExtractionResult.

PageCollections are the data source for extraction. A PageCollection loads the page source code for a specific language and one or more pages (Implementations: LiveWikipedia, DatabaseWikipedia).

getSource($pageTitle): returns the Wikipedia source code for the page $pageTitle

Collects RDFtriples while extraction is in process. Each wiki page needs an own instance of ExtractionResult.

Deletions:

To get the framework running on your local PC, it is recommended to start with the pre-configured files start.php or webStart.php. You will find these in the DBpedia root folder. The extraction code is available via sourceforge SVN (do not download the release version at sourceforge).
If you want to create your own dumps or work within your IDE or on the console, you should use start.php. If you first want to learn how DBpedia extraction works, or test /debug new extractors, webStart.php might best suit your needs, as it gives you a comfortable web debug interface, similiar to http://www4.wiwiss.fu-berlin.de/dbp-debug/. Make sure to download RAP – RdF API for PHP first if you want to use the webinterface.
If you intend to produce seperate output files for each Extractor, you will need an own ExtractionGroup for each extractor (as is done in the standard settings in start.php). Finally you will need to run your ExtractionJob through an instance of an ExtractionManager.

extractPage($pageID, $pageTitle, $pageSource): Includes the actual exraction process. Constructs a new ExtractionResult, extracts data from the sourcepage and stores the extracted data in the ExtractionResult. Is called for each page and must return an ExtractionResult.

PageCollections are the data source for extraction. A PageCollection loads the page sourcecode for a specific language and one or more pages (Implementations: LiveWikipedia, DatabaseWikipedia).

getSource($pageTitle): returns the Wikipedia sourcecode for the page $pageTitle

Collects RDFtriples while extraction is in process. Each wikipage needs an own instance of ExtractionResult.

2007-08-20 17:49:29

Comparison for /Documentation of 2007-08-20 17:49:29 and 2007-08-21 23:25:19

Additions:

Contents

1. Getting started

2. Functional overview

3. The Interfaces

3.1. Interface Destination

3.2. Interface Extractor

3.3. Interface PageCollection

3.4. Interface RDFnode

4. Essential Classes

4.1. Class ExtractionGroup

4.2. Class ExtractionJob

4.3. Class ExtractionManager

4.4. Class ExtractionResult

4.5. Class RDFtriple

2007-08-20 17:38:53

Comparison for /Documentation of 2007-08-20 17:38:53 and 2007-08-20 17:49:29

Additions:

The code extraction process is triggered via the ExtractionManager, which starts one or more ExtractionJobs. An ExtractionJob combines one or more ExtractionGroups with a PageCollection. The PageCollection is the data source, e.g. all articles from the Wikipedia SQL-Dump. ExtractionGroups consist of a Destination and one or more Extractors. Possible Destinations include your console, NTriple files and the web interface. Of course you are free to write your own destinations (e.g. databases, RDF/XML files).
Extractors are designed for single specific purposes. E.g. the InfoboxExtractor reads out information from Wikipedia Infoboxes. The ShortAbstractExtractor gets the first paragraph from an article and so on. DBpedia comes with extractors for many purposes already, though you are invited to add your own.
Extractors are the core of the data extraction process, as they parse and convert the Wikipedia pages. For each page, the extracted data is stored in an instance of ExtractionResult.
ExtractionGroups connect Extractors with Destinations. If you want to store the ExtractionResults from all Extractors in a single file, one ExtractionGroup is sufficient. Just create a new ExtractionGroup with a Destination and add the extractors you need.
If you intend to produce seperate output files for each Extractor, you will need an own ExtractionGroup for each extractor (as is done in the standard settings in start.php). Finally you will need to run your ExtractionJob through an instance of an ExtractionManager.
Destinations store extraction results. Included Destinations are NTriple files (NTripleDumpDestination), the console (SimpleDumpDestination) and a web interface (WebDebugDestination).

start(): Initializes the Destination (e.g. creates a new NTriple file). Is called once at the beginning of an ExtractionJob.
accept($extractionResult, $revisionID): Requires an ExtractionResult and a page revision. Reads out each triple from the ExtractionResult and prints it out or stores it in a file. Is called for each page.
finish(): Closes the destination. Is called once at the end of an ExtractionJob.
extractPage($pageID, $pageTitle, $pageSource): Includes the actual exraction process. Constructs a new ExtractionResult, extracts data from the sourcepage and stores the extracted data in the ExtractionResult. Is called for each page and must return an ExtractionResult.
finish(): Closes the extractor. Is called once at the end of an ExtractionJob.

Interface PageCollection:
PageCollections are the data source for extraction. A PageCollection loads the page sourcecode for a specific language and one or more pages (Implementations: LiveWikipedia, DatabaseWikipedia).
A PageCollection must have the methods:
URI, RDFliteral and RDFblankNode are implementations of RDFnodes.
Class ExtractionGroup
ExtractionGroups combine one or more Extractors with one Destination. If you want to create a single Dump for each Extractor in use, you will need to create a new ExtractionGroup for every Extractor, as an ExtractionGroup can hold only a single Destination.

__construct($destination, $metadestination = NULL): $destination is an object of a class, implementing the interface Destination. $metadestination is an optional Destination, where meta information can be stored. It is mainly used by the InfoboxExtractor, which collects all predicate names in a metdestination.

Class ExtractionJob
An ExtractionJob combines one or more ExtractionGroups (Extractors + Destination)
with one PageCollection (data source). ExtractionJobs are executed by the ExtractionManagaer.

addExtractionGroup($group): Adds a new ExtractionGroup to the job.

Class ExtractionManager
The ExtractioManager executes ExtractionJobs.
Cycling over all ExtractionGroups, the extraction manager first initializes the Extractors and the Destination (via their start() method).
Next, it iterates over all pages from a PageCollection and passes the page source to each Extractor, triggering its extractPage() method.
Finally it reads out the ExtractionResults from every Extractor and passes it to

execute($job): Starts the job as described above. Requires an object of class ExtractionJob.

Class ExtractionResult
Collects RDFtriples while extraction is in process. Each wikipage needs an own instance of ExtractionResult.

Deletions:

The code extraction process is triggered via the Extraction Manager, which starts one or more Extraction Jobs. An Extraction Job combines one or more Extraction Groups with a Page Collection. The Page Collection is the data source, e.g. all articles from the Wikipedia SQL-Dump. Extraction Groups consist of a Destination and one or more Extractors. Possible Destinations include your console, NTriple files and the web interface. Of course you are free to write your own destinations (e.g. databases, RDF/XML files).
Extractors are designed for single specific purposes. E.g. the Infobox Extractor reads out information from Wikipedia Infoboxes. The Short Abstract Extractor gets the first paragraph from an article and so on. DBpedia comes with extractors for many purposes already, though you are invited to add your own.
Extractors are the core of the data extraction process, as they parse and convert the Wikipedia pages. For each page, the extracted data is stored in an instance of Extraction Result.
Extraction Groups connect Extractors with Destinations. If you want to store the Extraction Results from all Extractors in a single file, one Extraction Group is sufficient. Just create a new Extraction Group with a Destination and add the extractors you need.
If you intend to produce seperate output files for each Extractor, you will need an own Extraction Group for each extractor (as is done in the standard settings in start.php). Finally you will need to run your Extraction Job through an instance of an Extraction Manager.
Destinations store extraction results. Included Destinations are NTriple files (NTripleDumpDestination), the console (Simple Dump Destination) and a web interface (Web Debug Destination).

start(): Initializes the Destination (e.g. creates a new NTriple file). Is called once at the beginning of an Extraction Job.
accept($extractionResult, $revisionID): Requires an Extraction Result and a page revision. Reads out each triple from the Extraction Result and prints it out or stores it in a file. Is called for each page.
finish(): Closes the destination. Is called once at the end of an Extracion Job.
extractPage($pageID, $pageTitle, $pageSource): Includes the actual exraction process. Constructs a new Extraction Result, extracts data from the sourcepage and stores the extracted data in the Extraction Result. Is called for each page and must return an Extraction Result.
finish(): Closes the extractor. Is called once at the end of an extraction job.

Interface Page Collection:
Page Collections are the data source for extraction. A Page Collection loads the page sourcecode for a specific language and or more pages (Implementations: Live Wikipedia, Database Wikipedia).
A Page Collection must have the methods:
URI, RDFliteral and RDFblankNode are implementations of RDFnodes.
Class Extraction Group
Extraction Groups combine one or more Extractors with one Destination. If you want to create a single Dump for each Extractor in use, you will need to create a new Extraction Group for every Extractor, as an Extraction Group can hold only a single Destination.

__construct($destination, $metadestination = NULL): $destination is an object of a class, implementing the interface Destination. $metadestination is an optional Destination, where meta information can be stored. It is mainly used by the Infobox Extractor, which collects all predicate names in a metdestination.

Class Extraction Job
An Extraction Job combines one or more Extraction Groups (Extractors + Destination)
with one Page Collection (data source). Extraction Jobs are executed by the Extraction Managaer.

addExtractionGroup($group): Adds a new Extraction Group to the job.

Class Extraction Manager
The Extractio Manager executes Extraction Jobs.
Cycling over all Extraction Groups, the extraction manager first initializes the Extractors and the Destination (via their start() method).
Next, it iterates over all pages from a Page Collection and passes the page source to each Extractor, triggering its extractPage() method.
Finally it reads out the Extraction Results from every Extractor and passes it to

execute($job): Starts the job as described above. Requires an object of class Extraction Job.

Class Extraction Result
Collects RDFtriples while extraction is in process. Each wikipage needs an own instance of Extraction Result.