Erlang and OTP in Action.pdf - Synrc

More documents

Recommendations

Info

288 12 Drivers and Multi-Language Interfaces At this point we have a really nice implementation of the cache. It already does a number if interesting things; things like provide a persistent store, distributes the cached objects, handles session objects, etc. Over all the Erlware guys are very happy. However, there is one small problem. Several of the objects that are cached are getting quite large. That’s making those objects a bit slow to distribute and they are taking up a lot of memory. After a short analysis it looks like these large chunks of data are pretty compressible. So Erlware guys put out a requirement that the cache support some type of compression. That’s generally not a problem, however, there aren’t really any good compression libraries in Erlang. There are a several very good compression libraries out there of course, but they are written in C though. it seems prudent to take one of those libraries and wrap it for use in Erlang rather than try to implement a compression scheme from scratch in Erlang. This approach also gives us the opportunity to introduce you to Erlang’s specialized foreign function interface. I said Erlang’s ‘specialized’ foreign function interface for a reason. At the highest level Erlang is no different from any other language; it allows you to make use of libraries and code written in other languages. However, as you dig deeper it becomes apparent that Erlang is actually very different. Most other languages have a foreign function interface to C. Where the languages provides some way to compile or links in C code that the language may, then call into. Erlang does it differently; it extends the message passing paradigm to interacting with foreign code as well. All of Erlang’s foreign interface approaches rely on passing messages back and forth between the foreign code and the Erlang system. The method of ©Manning Publications Co. Please post comments or corrections to the Author Online forum: http://www.manning-sandbox.com/forum.jspaforumID=454
289 transport and the ‘closeness’ between Erlang and the foreign code are really the only things that differentiate a C interface from one in Java or Python or Ruby. There are two types of foreign function interfaces in Erlang. These are the Port Driver and the Linked in Driver. The Port Driver and the Linked in Driver have similar high level semantics but very different low level semantics. The Port Driver method uses stdin and stdout to pass data to and from a running application via a “pipe”. Think of this pipe as exactly that, a conduit for passing information back and forth. Using this method of foreign language interface is very much language neutral. For example, Erlang can actually interface to any language you want, from shell scripts to assembly to scripting languages. If it can run and read from stdin and write to stdout Erlang can interface with it via Port Drivers. The other way, the Linked in Driver is much less flexible and safe for that matter. That is because it is exactly what its name implies, A Linked in Driver, a shared object that is compiled to native code that Erlang loads into its own address space. This has the benefit of being very fast, but a Linked in Driver has the potential to segfault the entire Erlang VM. This isn’t something that can happen when using a Port Driver by virtue of the fact that the Erlang VM and what it is communicating with via a pipe are different OS processes. In this chapter we are going to talk about both of these approaches. We will talk about how to use them, more importantly we will talk about when to use them. Both of these approaches have their own benefits and drawbacks. Understanding those benefits and drawbacks of each individual approach is key to choosing the right one to use in the context of your system. All this information will be used in the end to wrap a compression library for integration with the Simple Cache. 12.1 Port Drivers Port Drivers are the most common way to communicate with foreign code in Erlang. Port Drivers are actually rather simple. A Port Driver is just a program that can read from standard in and write to standard out. To use this program Erlang will start it up as a managed process and communicate with it over these channels. If the program being communicated with crashes then Erlang can just restart it and continue on its way (assuming of course, that you designed the program with this in mind). The program lives in its own process space and the only interaction it has with Erlang is over these standard io channels. That means that nothing the program does really has the ability to impact the running Erlang system. Considering the unsafe nature of most foreign code that’s a very big benefit. You can see in Figure 12.1 how communication with the remote code progresses. Figure 12.1 Communication Between Erlang and Remote Code ©Manning Publications Co. Please post comments or corrections to the Author Online forum: http://www.manning-sandbox.com/forum.jspaforumID=454
Page 2 and 3:
2 MEAP Edition Manning Early Access
Page 4 and 5:
4 1 The foundations of Erlang/OTP W
Page 6 and 7:
6 They are a bit like people: indiv
Page 8 and 9:
8 This may sound strange: shouldn
Page 10 and 11:
10 That’s all. (Although the spaw
Page 12 and 13:
12 affecting the rest of your code
Page 14 and 15:
14 simultaneous instances of C-like
Page 16 and 17:
16 2 Erlang Essentials This book is
Page 18 and 19:
18 42 2> What happened here was tha
Page 20 and 21:
20 CALLING Q() OR INIT:STOP() The s
Page 22 and 23:
22 User switch command --> j 1 {she
Page 24 and 25:
24 For division, there are two choi
Page 26 and 27:
26 in a system: currently, just ove
Page 28 and 29:
28 element 4 in this case), and the
Page 30 and 31:
30 That was the last in our list of
Page 32 and 33:
32 (See if you get the point before
Page 34 and 35:
34 the same atom for both. Do not a
Page 36 and 37:
36 % This is a comment and it ends
Page 38 and 39:
38 (You may have noticed in the cod
Page 40 and 41:
40 be reused in different places in
Page 42 and 43:
42 names like X1, X2, X3, for “mo
Page 44 and 45:
44 (The shell will print back the v
Page 46 and 47:
46 probably because you forgot to d
Page 48 and 49:
48 However, if the first clause doe
Page 50 and 51:
50 When a variable goes out of scop
Page 52 and 53:
52 2.7 - Funs We introduced funs br
Page 54 and 55:
54 for a long time (e.g., by storin
Page 56 and 57:
56 2.8.2 - Using try…catch In mod
Page 58 and 59:
58 The stack trace is a list, in re
Page 60 and 61:
60 2.9.2 - Mapping, filtering, and
Page 62 and 63:
62 you want to start working with b
Page 64 and 65:
64 This tells the compiler that we
Page 66 and 67:
66 -define(pair(X,Y), {X, Y}). Macr
Page 68 and 69:
68 -else. -endif. As the names indi
Page 70 and 71:
70 (If the monitored process identi
Page 72 and 73:
72 destination process P2 (regardle
Page 74 and 75:
74 ets:insert(T, {42, goodbye}) Now
Page 76 and 77:
76 o N-1 as the new value for N. Se
Page 78 and 79:
78 YOU CAN RELY ON TAIL CALL OPTIMI
Page 80 and 81:
80 might also make you remember som
Page 82 and 83:
82 When we are recursing over numbe
Page 84 and 85:
84 Now for the not_yet_implemented
Page 86 and 87:
86 Finally and mainly of historical
Page 88 and 89:
88 learned from Chapter 2 and used
Page 90 and 91:
90 When the function a process is s
Page 92 and 93:
92 The third and final constituent
Page 94 and 95:
94 section here from how to documen
Page 96 and 97:
96 attributes. They are highlighted
Page 98 and 99:
98 Function Name Associated behavio
Page 100 and 101:
100 QUICK MESSAGING REFRESH Process
Page 102 and 103:
102 that we can message this proces
Page 104 and 105:
104 callback exported which is tr_s
Page 106 and 107:
106 will be picked up by tr_server:
Page 108 and 109:
108 brings us to the other clause o
Page 110 and 111:
110 The return value from the appli
Page 112 and 113:
112 I use telnet to connect via TCP
Page 114 and 115:
114 for handling code within applic
Page 116 and 117:
116 code:priv_dir(). This will retu
Page 118 and 119:
118 mod This is the place where you
Page 120 and 121:
120 colliding is your own. Unfortun
Page 122 and 123:
122 %% Supervisor callbacks -export
Page 124 and 125:
124 propagate the failure up the su
Page 126 and 127:
126 5 Processes, Linking and the Pl
Page 128 and 129:
128 5.1.2 Shared Memory With Locks
Page 130 and 131:
130 Another approach to concurrency
Page 132 and 133:
132 Of course, you don't get this f
Page 134 and 135:
134 figure 5.1 - cascading link fai
Page 136 and 137:
136 time to time. In Listing 5.2, y
Page 138 and 139:
138 addition is at code annotation
Page 140 and 141:
140 Go ahead and make the relevant
Page 142 and 143:
142 see this example in production
Page 144 and 145:
144 this allows the VM to do is ele
Page 146 and 147:
146 6 Implementing a Caching System
Page 148 and 149:
148 To put this functionality in pl
Page 150 and 151:
150 Getting into more detail about
Page 152 and 153:
152 First we define our module name
Page 154 and 155:
154 Restart = temporary, Shutdown =
Page 156 and 157:
156 {start_phases, []}]}. This app
Page 158 and 159:
158 ]). -export([init/1, handle_cal
Page 160 and 161:
160 At annotation number 1 we captu
Page 162 and 163:
162 handle_call(fetch, _From, State
Page 164 and 165:
164 implementing the sc_store modul
Page 166 and 167:
166 call. Passing the table handle
Page 168 and 169:
168 quite incidental and unimportan
Page 170 and 171:
170 SIMPLE_CACHE:LOOKUP/1 Lookup is
Page 172 and 173:
172 7 Logging and Eventing the Erla
Page 174 and 175:
174 is mostly for you, the develope
Page 176 and 177:
176 used in quite a few application
Page 178 and 179:
178 {noreply, State}. terminate(_Re
Page 180 and 181:
180 {links,[,]}, {dictionary,[]}, {
Page 182 and 183:
182 gen_event the functions that we
Page 184 and 185:
184 -> io_lib:fwrite("WARNING ~p",
Page 186 and 187:
186 When a message (event) is sent
Page 188 and 189:
188 replace(Key, Value) -> gen_even
Page 190 and 191:
190 -behaviour(gen_event). -export(
Page 192 and 193:
192 8 Introducing Distributed Erlan
Page 194 and 195:
194 my computer Process 1 messages
Page 196 and 197:
196 Figure 8.5 shows the same nodes
Page 198 and 199:
198 Eshell V5.6.2 (abort with ^G) (
Page 200 and 201:
200 on the receiving end at it’s
Page 202 and 203:
202 You should see the text represe
Page 204 and 205:
204 --> c 2 Eshell V5.6.5 (abort wi
Page 206 and 207:
206 Resource Instance Resource A re
Page 208 and 209:
208 target_resource_types = [], loc
Page 210 and 211:
210 trade_resources() -> gen_server
Page 212 and 213:
212 8.4 - Summary We have really co
Page 214 and 215:
214 Architecture of Our Cache As yo
Page 216 and 217:
216 SYNCHRONOUS MESSAGING Synchrono
Page 218 and 219:
218 providing, or holding up the cl
Page 220 and 221:
220 session data. In this case the
Page 222 and 223:
222 this database you will become f
Page 224 and 225:
224 Mnesia requires this thing call
Page 226 and 227:
226 Listing 9.4 - Creating Our Tabl
Page 228 and 229:
228 keys can be repeated. Records t
Page 230 and 231:
230 With our data now in the databa
Page 232 and 233:
232 4. delete/1 Init is going to be
Page 234 and 235:
234 This is defined as one of the c
Page 236 and 237:
236 lists:foreach(fun(N) -> net_adm
Page 238 and 239: 238 9.3.3 - Integrate resource disc
Page 240 and 241: 240 9.3.4 - Bring the Mnesia Tables
Page 242 and 243: 242 We have now built a system that
Page 244 and 245: 244 going to run and do interesting
Page 246 and 247: 246 All of this information is impo
Page 248 and 249: 248 you can tie these living chunks
Page 250 and 251: 250 applications. The figure illust
Page 252 and 253: 252 {erts,"5.7.2”}, [{kernel,"2.1
Page 254 and 255: 254 calles lib somewhere on the fie
Page 256 and 257: 256 As you can see all of the appli
Page 258 and 259: 258 -config ../releases/simple_cach
Page 260 and 261: 260 10.4 - Conclusion You now know
Page 262 and 263: 262 3. Create RESTful interface app
Page 264 and 265: 264 1. spawn handler 3. call for ne
Page 266 and 267: 266 ti_sup:start_child(), {ok, Pid}
Page 268 and 269: 268 -define(SERVER, MODULE). -recor
Page 270 and 271: 270 request is sent, this effective
Page 272 and 273: 272 | | `-- restful_interface.app |
Page 274 and 275: 274 TWO WAY RADIO STATE MACHINE A s
Page 276 and 277: 276 clauses If you get that then we
Page 278 and 279: 278 User-Agent: curl/7.16.3 (powerp
Page 280 and 281: 280 When you look at this the state
Page 282 and 283: 282 ok; terminate(_Reason, _StateNa
Page 284 and 285: 284 below which is called from the
Page 286 and 287: 286 The get entry point is, perhaps
Page 290 and 291: 290 Of course, there is no such thi
Page 292 and 293: 292 want to get the exit_status as
Page 294 and 295: 294 sys.stdout.write(line) #3 sys.s
Page 296 and 297: 296 Now that we have all of the ‘
Page 298 and 299: 298 Figure 12.2 - 5 instances shari
Page 300 and 301: 300 The Erlang Virtual Machine comm
Page 302 and 303: 302 control timeout process ready_a
Page 304 and 305: 304 ERL_DRV_BINARY, (ErlDrvTermData
Page 306 and 307: 306 This doesn’t mean that it’s
Page 308 and 309: 308 In this particular example, com
Page 310 and 311: 310 NULL, /* F_PTR init, N/A */ drv
Page 312 and 313: 312 As before the main thing to be
Page 314 and 315: 314 if (ei_decode_atom(buff, &index
Page 316 and 317: 316 long data_len; if (ei_decode_lo
Page 318 and 319: 318 } "Expecting atom 'compressed'
Page 320 and 321: 320 but there are a few, very disti
Page 322 and 323: 322 12.16 The Compression/Decompres
Page 324 and 325: 324 lookup(Key) -> sc_event:lookup(
Page 326 and 327: 326 “HBase is the Hadoop database
Page 328 and 329: 328 perform the same function. Howe
Page 330 and 331: 330 OtpErlangObject o = mbox.reciev
Page 332 and 333: 332 contains the PID of the sender
Page 334 and 335: 334 13.2.1 - Installing HBase The f
Page 336 and 337: 336 error end. The third part of ou
Page 338 and 339:
338 public SimpleCacheHBaseMain(Str
Page 340 and 341:
340 message that we didn’t know h
Page 342 and 343:
342 private void doGet() throws IOE
Page 344 and 345:
344 NavigableMap that will describe
Page 346 and 347:
346 The second function that we nee
Page 348 and 349:
348 14 Optimization and Performance
Page 350 and 351:
350 Realistic Timely expect to, ach
Page 352 and 353:
352 Below in code listing 15.2 is a
Page 354 and 355:
354 Actually looking at the Pids fo
Page 356 and 357:
356 Ruling those two things out we
Page 358 and 359:
358 run_with_fprof() -> fprof:trace
Page 360 and 361:
360 is the refactoring bottle necks
Page 362 and 363:
362 However, its very useful to kno
Page 364 and 365:
364 inefficient both in terms of sp
Page 366 and 367:
366 ** exception error: bad argumen
Page 368 and 369:
368 all memory available and finall
Page 370 and 371:
370 The preceding function will be
Page 372 and 373:
372 15 Make it Faster “There is n
Page 374 and 375:
374 atom name and you have a recipe
Page 376 and 377:
376 example of where this knowledge
Page 378 and 379:
378 SIZE/1 VS LENGTH/1 This is a so
Page 380 and 381:
380 receive Msg -> handle_msg(Msg),
Page 382 and 383:
382 Function invocation type Local
Page 384 and 385:
384 Profiling and measuring is the
Page 386 and 387:
386 an addition of the total time s
Page 388 and 389:
388 both processes we can see that
Page 390 and 391:
390 run_with_fprof() -> fprof:trace
Page 392 and 393:
392 implementation is the best prac
Page 394 and 395:
394 A.2.2 - Resolving configuration
Page 396 and 397:
396 Appendix B Lists and referentia
show all

Erlang and OTP in Action.pdf - Synrc

Create successful ePaper yourself

Delete template?

Save as template?