Grid Computing Cluster â the Development and ... - Lim Lian Tze

More documents

Recommendations

Info

$LaTeX: More Than Just Academic Papers and Theses - Lim Lian Tze$

Image: Binary data everywhere. Illustration by ©Rodolfo Clix (http://www.sxc.hu/profile/clix).DYNAMIC REPLICA MANAGEMENTIN DATA GRID ENVIRONMENTGrid Computing LabSchool of Computer SciencesTEAM MEMBERSProject Leader : AP Chan Huah YongResearchers : Aloysius Indrayanto, Muhammad Muzzammil bin Mohd SalahudinContributors : Siamak Sarmady, Wong Jik Soon, Seow Kwen Jin, Zeinab Noorian, Ang Sin Keat, Cheng Wai KhuenINTRODUCTIONAccessing, managing, querying, andprogramming a database system arenormally dependent on the type ofthe database deployment, standards,conventions defined by the databasevendor, and the language that will beused to define the query. Currently,the Relational Database ManagementSystem (RDBMS) is one of the mostwidely used database system. Queryon these kind of databases are usuallydone by using a language namedStructured Query Language (SQL).RDBMS is currently implementedand supported by many different vendors.Every vendor has its ownmethod or convention on how usersshould access its database system.This convention is defined as a setof programming procedure calledApplication Programming Interface(API). These differences would causea developer (programmer) the needto learn many different API if he/shewants to use and integrate RDBMS deploymentsfrom multiple vendors.SQL is a standardized language.However, the standard set of SQL commandsnormally are not enough fora vendor. Hence, database vendorswould normally add non-standard extensionsto the SQL so that users canuse their RDBMS more efficiently. Thisextension would cause an administratoror a developer the need to learnmany non-standard SQL commands.In addition, some vendors also providea custom API which will need tobe used if a developer wants an evenmore efficient access to the RDBMS.Some vendors only provide customAPIs to access the RDBMS’ specific features(no extension to the SQL language).Learning many different APIs andnon-standard SQL extensions wouldnot be efficient for a developer. Timecould be wasted to factorize whatcan be done and what cannot bedone with a particular RDBMS product/deployment.Hence, interoperatingRDBMS from multiple vendorswould be a maintenance problem.Performing a custom distributedquery between two different RDBMSthat are installed in two differenthosts (computers) would be impossibledue to the fact that the vendorsuse their own proprietary protocolsfor communication betweentheir RDBMS servers and clients. Also,not all RDBMS vendors support distributedquery between two (or more)hosts. In addition, the cost of thesoftware license and deployment forthese kind of databases are expensive.A large database vendors, such asOracle and IBM DB2, provides supportsfor distributed query. Hence,their system is called DistributedDatabase System (DDS). However, thetwo systems may not cooperate wellwith each other. Hence, performinga distributed query between Oracleand DB2 would be difficult.The VDDBMS project aims to providea compatibility layer betweendifferent RDBMS implementations andthe end users. The main objective isto allow distributed queries betweentwo or more RDBMS from differentvendors with the same SQL and API.14
Grid Computing Cluster – the Development and Integration of Grid Services and ApplicationsFigure 1: Current architecture of the VDDBMS systemMETHODOLOGYThe methodology applied by theproject to achieve its goal can bebriefly described as follow:1. An SQL parser has been implementedto parse the user-given SQLstatements into tokens (symbolsand data).2. A resource locator has been implementedto add references to thetokens to record the hosts (datasources) on which the SQL wouldneed to be executed.3. A token splitter has been implementedto split the tokens into subtokens.These steps are necessarybecause a data source does not haveto execute the complete tokens.4. An instruction sequencer has beenimplemented to execute to subtokensin the appropriate datasources in the correct sequence.5. A SQL generator has been implementedto translate back the subtokensinto SQL statements understandableby the target datasources.6. A post-processor has been implementedto aggregate back the subresultsfrom the data sources intothe final result.DESIGN & IMPLEMENTATIONThe system can be separated into fourmain modules:1. Core engine;2. Backend drivers for each RDBMSvendor to be supported;3. Language binding;4. Network server and client drivers.The system is mainly developedusing C++ in Linux. Figure 1 showsthe current architecture of the system.The core engine interacts with thebackend RDBMS using the specificdrivers. The core engine is workingtogether with resource catalogue andlock manager to process user requests.User can connect to the VDDBMS systemeither locally or remotely. Currentlylocal connection is supportedby native API in C++ and PHP. Remoteconnection is supported by networkAPI in C++, PHP, Java, and .NET.Figure 2 shows the modules in theVDDBMS system with their simplifieddataflow. In the figure the solid linesdefines communication path whichare done using API. The dashed linedefines communication path whichare done using network protocol.Most of the API communicationpaths are private. Only the C++ andPHP public interfaces are usable bythe user. Hence, changes in the internalAPI would not raise incompatibilityissue with user applications.The network (VDDBMS) server andthe client drivers (C++, Java, .NET)are completely independent. As longas the same protocol is used, changesin the server would not affect theclient. The protocol has been designedto be extensible. Hence, anewer server would be able to manageolder client driver implementations.Figure 3 shows the high level flowof the VDDBMS engine while Figure 4shows the detailed flow of the coreexecution engine.The main features of the currentsystem are:1. Data replication and horizontalfragmentation.2. Support for simple SQL INSERT,DELETE, UPDATE, andSELECT queries.3. Prepared statement for SQLINSERT.4. Native DATE/TIME field support.5. Stored procedure with supportfor looping, conditional statements,simple calculations, and atomic integeroperations.6. MySQL, PostgreSQL, Microsoft-SQL, and Oracle are supported asthe backend RDBMS (data sources).SUB-PROJECTS 15
Page 1: Grid Computing ClusterThe Developme
Page 4: © 2009 Grid@USMPlatform for Inform
Page 8: Image: It’s a wide, yet connected
Page 11 and 12: Grid Computing Cluster - the Develo
Page 13 and 14: PROJECT MILESTONESPast events2007Pl
Page 15 and 16: Image: Eye on Malaysia by Titiwangs
Page 19: Grid Computing Cluster - the Develo
Page 29 and 30: Image: S/Cultura Luminosa bookshelv
Page 33 and 34: Image: A business handshake. Photog
Page 37 and 38: Image: A programmer hard at work. P
Page 41 and 42: Image: Ethernet crossover cable. Ph
Page 51 and 52: Image: Fireworks. Photograph by ©G
Page 57: Grid Computing Cluster - the Develo

Grid Computing Cluster â the Development and ... - Lim Lian Tze

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?

Grid Computing Cluster â the Development and ... - Lim Lian Tze