An Introduction to the Progress Database - Radius

An Introduction to the 

Progress Database 

David Johnson 

Senior Solution Engineer 

June 20, 2007

Agenda 

• Database Basics & Architecture 

• Creating, Starting, Stopping a Database 

• Backup & Restore 

• Dump & Load 

• Basic Tuning 

• Monitoring & Replicating Data 

2 

© 2007 Progress Software Corporation

What Is a Database Management System 

• Software that 

• Organizes and integrates data 

• Shares data 

• Provides the ability to 

– Create, 

– Retrieve, 

– Update, 

– and Destroy data contained in the database 

• Manages the storage devices that contain data 

• Handles error recovery when something goes 

wrong 

3 


Objectives of an RDBMS 

• Quick & Efficient access to Entity Relational Data 

A 

F 

B 

S 

R 

B 

O 

C 

U 

B 

O 

X 

G 

L 

A 

Z 

X 

U 

E 

R 

Y 

C 

S 

S 

A 

F 

B 

X 

B 

O 

A 

Y 

C 

O 

R 

Z 

A 

X 

B 

Y 

C 

Z 

Data On 

Disk 

Information 

In Memory 

Knowledge 

On Client 

4 


Why use a RDBMS 

• Lets you take for granted the following: 

• Find what you need quickly 

• Allow shared access to the data 

• Coordinate concurrent updates to the same 

records 

• Enforce security rules 

• Ensure structural integrity 

• Provide high performance 

• Correct the data when errors occur 

• Perform crash recovery 

• Map data structures to compact storage 

• Add new data definitions on-line 

• And more… 

5 


Database Storage 

Logical View 

• Area: A group of database objects 

(tables, indexes) 

• Table: A set of records (rows) that are 

composed of fields (columns) 

• Field: A single data value (or array) 

• Index: An ordered list of key value-rowid pairs 

that identify all rows in a table 

• Index entry: one key value-rowid pair 

6 


Storage Areas (Progress V9) 

Logical 

View 

Table A 

Table B 

Index A-1 

Index B-1 

Table C 

Table D 

Index C-1 

Index D-1 

Index E-1 

Table E 

Index E 

Area 6 Area 7 Area 8 

Area 51 

Physical 

View 

Extent 

Extent 

Extent 

Extent 

Extent 

Extent 

Extent 

Extent 

Extent 

Extent 

Disk Storage - Files or Raw Slices 

7 


Database Storage 

Physical View 

• Block: A fixed-size unit of disk I/O and storage. All 

data areas have same block size. 

• Extent: A unit of disk storage 

• Either a file or a raw disk slice 

• May be fixed or variable size 

• Area: A group of physical storage objects 

8 


Database Blocks within Extents 

Fixed extent 

Cluster 

Database block 

Variable extent 

Not yet 

allocated 

by O/S 

Filled Partly filled Free Not yet allocated 

9 


Transactions – What Are They 

• Transactions are an error handling and 

recovery mechanism. If nothing goes wrong, 

you don’t need them. 

• Eventually, something will go wrong. 

• When something does go wrong, or you say 

“Ooops, I don’t want to do this after all.” 

10 

The transaction manager graciously puts 

everything back the way it was, just as if 

nothing had ever happened. 


Database Transaction Logs 

• Progress generates two types of transaction 

logs: 

• The undo-redo log or Before-Image Log contains 

a record of recent changes. 

• The redo log or After-Image Journal contains a 

record of all changes since a complete backup of 

the database was made. 

• The records (also called “notes”) that are written 

to the two transaction logs are identical. 

• Other database systems use similar 

techniques 

11 


Database Transaction Logs 

BI 

Undo-Redo Log 

(Before-Image Log) 

Recorded Changes 

Original 

Database 

db 

Transactions 

make changes 

db’ 

Updated 


AI 

Redo-Log 

(After-Image Journals) 

Recorded Changes 

12 


Progress RDBMS Architecture 

Clients 

WebSpeed 

4GL Client 4GL Client 

AppServer 

ODBC 

JDBC 

ESQL/C 

Servers 

4GL 


Server 

OpenEdge-SQL 


Server 

Shared Database Storage Engine 

13 


Agenda 







14 


Methods to Create a Database 

‣ Using PRODB 

‣ Using a .ST file, PROSTRCT CREATE, 

and PROCOPY 

‣ Data Dictionary tool 

‣ Data Administration tool 

‣ PROREST 

‣ PROUTIL CONV89 | CONV910 

15 


Using a .ST File, PROSTRCT CREATE, and 

PROCOPY 

1 

2 

PROSTRCT CREATE 

Void DB 

Source DB 

.ST File 

3 

PROCOPY 

Resultant DB 

16 


Structure Description File 

• # Sampledb.st file 

• b .\Sampledb.b1 f 32000 

• b .\Sampledb.b2 f 32000 

• b .\Sampledb.b3 

• # 

• d "Schema Area":6,32 .\Sampledb.d1 

• # 

• d "Employee":7,32;64 .\Sampledb_7.d1 f 32000 

• d "Employee":7,32;64 .\Sampledb_7.d2 

• # 

• d "Inventory":8,32;64 .\Sampledb_8.d1 f 64000 

• d "Inventory":8,32;64 .\Sampledb_8.d2 

• … 

• a .\Sampledb.a1 f 32000 

• a .\Sampledb.a2 f 32000 

• a .\Sampledb.a3 

17 


Data Dictionary & Data Administration 

18 


Starting a Database Broker 

proserve db –H host –S service 

Broker 

Shared Memory 

19 


Starting up Multiple Brokers 

Primary broker 

proserve db –Mn n 

SQL 

Broker 

Shared Memory 

Additional broker 

proserve db –m3 

4GL 

Broker 

20 


Using a Parameter File 

NEWDB.PF 

proserve –pf newdb.pf 

-db newdb 

-H localhost 

-S 3000 

-n 101 

-Mn 15 

-Ma 5 

-Mpb 4 

-ServerType SQL 

… 

SQL 

Broker 

4GL 

Broker 

Shared Memory 

21 


Shutting Down a Database in the Multi-User 

Environment 

‣PROSHUT 

‣Can also be used to disconnect a user 

‣PROMON 

22 


The Progress Explorer Tool 

23 


Agenda 







24 


Backup & Restore - Planning 

‣ Which files of the database do we need to back up 

‣ Which backup utility do we use 

‣ What impact would our backup method have on our 

users 

‣ How much data should we back up 

‣ What is the cost of our backup and can we afford it 

‣ What kind of backup media should we use 

‣ When and how often should we back up 

‣ How do we label and archive the backups 

‣ When and how do we test the backups 

25 


OpenEdge RDBMS Backup Utility 

probkup [ online ] db-name 

[ incremental ] device-name 

Required files 

.DB 

.Dn 

.Bn 

Full backup or… 

Incremental backups 

26 


Verify the Backup 

prorest db-name backup-file -vf 

Backup file 

.BCK 

27 


Restore the Backup 

prorest db-name backup-file 

Backup file 

.BCK 

28 


Agenda 







29 


Reasons to do a Dump & Load 

• Create a new version of a database 

• Economize disk space 

• Convert a database to a new OpenEdge 

release or a different operating system 

platform 

• Load updated data definitions to upgrade a 

database schema 

• Change the database block size 

30 


Storage after “months” of database activity 

extent 1 (fixed) 

Type I Storage Area 

extent 2 (variable) 

Reserved Blocks 

Table 1 data 

Table 2 data 

Table 3 data 

Table 4 data 

Table 5 data 

Table 6 data 

1 Database block 

High 

Water 

Mark 

Total 

Blocks 

31 


Dump and Load Process 

proutil db-name –C dump table-name target-dir 

Dump 

Load 

Source Database 

Definitions 

Target Database 

Data 

proutil db-name –C load .bd-file 

32 


Dump & Load - Database Administration Tool 

Source Database 

Data.d 

33 


OpenEdge 10 

Type II Storage 

Type I Storage Area 

Type II Storage Area 

Db extent (variable) 

Db extent (variable) 

High 

Water 

Mark 

Total 

Blocks 

High 

Water 

Mark 

Total 

Blocks 

Area Control Object 

Table 1 data 

Table 2 data 

Table 3 data 

1 Database block 

34 


Agenda 







35 


Progress Database Tuning 

Server startup 

• Buffer Pool Size 

• -B, Increase when buffer hit ratio < 90% 

• Spinlock Retry Count 

• -spin = 50,000 (rule of thumb), Enterprise only 

• Start BI Log Writer (BIW) 

• 1 per database, Enterprise only 

• Start Asynchronous Page Writer (APW) 

• Increase until 0 blocks flushed at checkpoint, Ent. only 

• Increase BI Log Block Size 

• -biblocksize, set to 8 

• Add BI Log Buffers 

• -bibufs, Default = 5, set in range of 25-50 

36 


Progress Database Tuning 

Server startup cont… 

• Increase BI Log Cluster Size 

• -bi 4096 

• Move Data Extents to Striped Array 

• Move BI Log to a Separate Disk 

Client side 

• -T (Temp files) 

• -q (quick request) 

• -TM & -TB sort options 

37 


Example Tuning Results 

throughput (tps) 

800 

700 

600 

500 

400 

300 

200 

100 

0 

30 

no tuning 

581 

simple tuning 

38 


Agenda 







39 


Monitoring using Promon 

PROGRESS MONITOR Version 9 

Database: /FULLPATH/DBNAME 

1. User Control 

2. Locking and Waiting Statistics 

3. Block Access 

4. Record Locking Table 

5. Activity 

6. Shared Resources 

7. Database Status 

8. Shut Down Database 

T. Transactions Control 

L. Resolve Limbo Transactions 

C. Coordinator Information 

M. Modify Defaults 

Q. Quit 

Basic information 

can be found on screen 5 

Also look under R&D 

Go ahead and poke 

around; just don’t 

choose “Shut Down” 

40 


Key performance activity metrics 

41 


Other Tools For Database Analysis 

Tool 

• Prostrct statistics 

• Prostrct list 

• Proutil dbanalys 

• statParse.p 

• VST’s 

• Top 

• Your favorite OS tool 

Frequency 

(monthly) 

(as needed) 

(weekly or monthly) 

(monthly) 

(program dictated) 

42 


Monitoring Using Fathom Management 

43 


Data Replication - Fathom Replication Plus 

44 


Progress and Radius Solutions 

Reliable, Scalable, Efficient, Open, Cost-Effective… 

Progress Software has over 20 years of proven 

success in partnering with ISVs to ensure the 

fastest, most cost-effective path for delivering 

competitive business applications. 

IDC 

“…Progress partnering approach leads the industry” 

Forrester reports… 

“…40% cost savings on the Progress platform versus an alternative platform" 

45 


46 


Why Upgrade to OpenEdge 

Large Database Support 

64 bit Rowids 

64 bit Sequences 

64 bit Integer Datatype 

Large Index Key Entries (1970) 

32,000 areas 

Performance 

Type II Storage Areas 

Increased addressable shmem 

Larger Shmem segment support 

Internal protocol enhancements 

Buffers, Locks, Indexing 

Auto Record Defrag 

High Availability 

Online Schema changes 

Online space management 

Enabled/Disable ai online 

Security 

SSL 

Auditing 

Datatype Support 

BLOB, CLOB 

Date/Time, Date/Time w/tz 

int 64 

Maintenance 

.st file syntax checker 

AI management 

More online maintenance 

Multi threaded utilities 

Log File 

New format 

Significant events 

Improved log file management 

Db I/O by User by Object 

47

An Introduction to the Progress Database - Radius

Create successful ePaper yourself

Delete template?

Save as template?