Enhancing LAMMPS Capabilities

Enhancing LAMMPS CapabilitiesGranular Models, Coding Concepts, CAD Interoperabilityand Coupling to Continuum Methods2nd LAMMPS Workshop, Albuquerque, Aug 2011Christoph Kloss*, Christoph Goniva**, Stefan Amberger,Alice Hager, Andreas Aigner, Michael Friedl, Stefan Pirker****christoph.kloss@jku.at, ** christoph.goniva@jku.at, ***stefan.pirker@jku.atall CD Laboratory on Particulate Flow ModellingJohannes Kepler University Linz, Austriawww.liggghts.com | www.cfdem.com | www.particulate-flow.at

Outline

Open Source DEM and CFD-DEMOutline1. IntroductionOur Group, Mission Statement2. ModelsGranular Contact Models, Heat Transfer, SPH, Multisphere Model3. CAD InteroperabilityMesh Import, Particle Insertion, Mesh regions, Wear Prediction4. Features and Coding ConceptsLoad Balancing, Per-Particle Properties, Transport Equations5. Coupling to Continuum MethodsOpenFOAM®, Resolved and Unresolved CFD-DEM, DEM MultiphysicsChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 3

I. IntroductionAbout Us / Mission Statement

About UsMission StatementCD Lab Particulate Flow Modelling (head: Stefan Pirker)6P Post-Docs, tD 7 PhD students, t 5 master studentstGoal: Particulate Flow Modelling...Research focus:• Fluid flow CFD, LB, SPH• Granular flow CFD, DEM/MD• Heat transfer• Experimental Validation...of industrially relevant processesApplication examples include:Blast furnaces, cyclones, pneumatic conveying, fluidized beds, dryers, wet scrubbers,conveyor and chute systems, tablet pressing, powder sintering, hopper flow,soil sampling, energy storage, river bed erosion, blood flows...Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 5

The LIGGGHTS/CFDEM CodesOverviewLIGGGHTS = An Open Source, C++, MPI parallel DEM codeLAMMPS IMPROVED FOR GENERAL GRANULAR AND GRANULARHEAT TRANSFER SIMULATIONSCFDEM = Coupling of LIGGGHTS to CFD code OpenFOAM®CFD-DEMWWW.LIGGGHTS.COM | WWW.CFDEM.COMThe web platform now has about 600 registered usersPublished in [1, 2, 3, 4, 10, 11, 12, 13, 14, 16]Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 6

The LIGGGHTS+CFDEM CodesCooperationsD Lab CDFurther (formal and informal) cooperations for LIGGGHTS and CFDEMChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 7

The LIGGGHTS CodeRelation to LAMMPSLIGGGHTSbeta 1LIGGGHTS1.018.04.201019. Feb. 2010 Currently ~30.000lines of code (LOC)...LIGGGHTS1.3.115.06.2011presentLIGGGHTS2.0???LAMMPSLAMMPSLAMMPS....24-Jan-2010 30-Mar-201015-Jun-2011Sandia National Labs, Albuquerque, US+ community contributionsChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 8

II. Models

LIGGGHTS Models - ReleasedSPH – Smoothed Particle HydrodynamicsSPH is well-suited for geometrically complex systems, and cases wherenumerical diffusion i of Finite it Volume approach is not acceptableReleased in LIGGGHTS 1.4 – commands pair sph, fix sph/density/continuity,fix sph/density/corr, fix sph/pressure/summation, fix wall/sph, fix wall/region/sphPrinciple of SPHSloshing Tank with SPH in LIGGGHTS vs.Volume of Fluid (VOF) Method in OpenFOAM®Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 10

LIGGGHTS Models - ReleasedHertz-Mindlin (HM) Granular Contact ModelF = ( k δn− γ vn ) + ( k δ t − γ vt )n ij{n ij{t ij{t ij{normal normaltangential tangentialoverlap relative vel .overlap relative vel .14444244443normalforce14444244443tangentialforcekγknnt43* *= YRδn,5= −2β Snm6* *= 8GRδ5γ t= β m6*−2 S t≥n,*≥ 0,0.* *Sn = 2YR δn, St= 8G1*Rδn22( 1−ν) ( 1−ν)ln( e ) 112β = , = + ,22*ln ( e ) + π Y Y1Y2G*1R2=1=R( 2+ ν)( 1−ν) 2( 2+ ν)( 1−)1 12ν2Y1+R11= ,* *1 2 m+1=mY1+mY...Young's modulus G...Shear modulusν...Poissonνratioe...coeff.ofofrestitution122*Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 11

LIGGGHTS Models – UnreleasedParticle BondsParticle bonds are inter-particleconnections able to resist• tension/compression,• shear,• torsion and• bendingup to a certain breakage limit. Once broken, the bond connection islost permanently. Particle bonds and related models can be used tomodel soils, „glued particles“, particle breakage, crack formation inbeams („lattice beam model“) etc...Looking for co-workers to make model ready for release!Asaf et. al, Soil & Tillage Research 92 (2007) 227–242 “Determination of discrete element model parameters required for soil tillage”Shmulevich: Soil & Tillage Research 111 (2010) 41–53 “State of the art modeling of soil–tillage interaction using discrete element method”Zhang, Li: Journal of Terramechanics 43 (2006) 303–316 “Simulation on mechanical behavior of cohesive soil by Distinct Element Method”,Pics from: Latham, S, Weatherly, D.,: Scripting Parallel Discrete Element Simulations with ESyS_Particle, https://twiki.esscc.uq.edu.au/bin/view/ESSCC/ESySParticleDownloadChristian Doppler Laboratory on Particulate Flow Modellingwww.particulate-flow.at13

Lattice Beam ModelDynamic fracture roughness in a concrete microstructureLIGGGHTS will incorporate a beam latticemodel (e.g. *,**) used to study fracture andfragmentation ti phenomena.* G. Lilliu, J.G.M. van Mier, 3D lattice type fracture modelfor concrete, Engrg. Fract. Mech. 70 (2003) 927–941.**J.G.M. van Mier, E. Schlangen, A. Vervuurt, Lattice typefracture models for concrete, in: H.B. Mühlhaus (Ed.),Continuum Models for Materials with Microstructure,John Wiley & Sons, 1995, pp. 341–377.Jean-Francois JerierEPF LausanneComputational SolidMechanicsNumerical concrete samplebased on a X-ray microtomographyimage (25mm 3 )The crack paths (red)crossing the sampleTimoshenko beam theory leads to theinteraction between the spheres500.000000 polydisperse spheres and 4 million beams represent this sampleEach simulation calculated on 60 processors in 3 daysChristian Doppler Laboratory on Particulate Flow Modellingwww.particulate-flow.at

LIGGGHTS ModelsCapturing non-sphericityGranular particles outside laboratories are rarely spherical≠Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 15

LIGGGHTS ModelsCapturing non-sphericityMulti-Sphere Approach• Step 1 – Particle Image by Laser Scanner• Step 2 – Generate Multi-Sphere Simulation ModelChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 16

LIGGGHTS ModelsCapturing non-sphericity• Step 3 – LIGGGHTS SimulationShow case cylinder: Gravity tilted so that angle of repose formsSphericalNon-SphericalChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 17

LIGGGHTS ModelsCapturing non-uniformityIn reality, no particle is like any other...• Fix particledistribution/discrete takes particletemplates as inputand serves as input for insertion commands• Fix particletemplate/sphere and fix particletamplate/multispherefix particledistribution/discreteFixParticletemplate **list;list[0]list[1]list[2]fix particletemplate/sphere fix particletemplate/sphere fix particletemplate/multisphere50 mass%ρ = 2000 kg/m³-uniform size distribution15 mass%ρ = 1500 kg/m³35 mass%ρ = 3500 kg/m³Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 18

III. CAD Interoperability

CAD InteroperabilityImporting a MeshIndustrial scale granular problems need CAD/Mesh interface!Rotary dryer, ~1M particlesWork with Diego Peinado(Intrame Madrid)Command fix mesh/granreads a triangular mesh fromSTL or VTKCommand fix move/mesh/grantranslates/rotates etc the mesh(like fix move)Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 20

CAD InteroperabilityGenerating a Non-trivial Particle PackingPacking in tetrahedral meshCommand region tetmeshReads tet mesh from VTK to beused as a regionCommand fix insert/packWorks similar to fix pourCan use arbitrary regions(also region tetmesh)Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 21

CAD InteroperabilityGenerating a Particle StreamParticle insertion at surfaceCommand fix insert/streamExtrudes triangular surface meshvelocityParticles are inserted in packageslike with fix pour and are integratedby the fix with constant vel untilthey reach the surface mesh. BCsare enforced at the surface mesh.Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 22

Industrial ApplicationWear Prediction at Transfer ChuteWork with Andre Katterfeld (Univ. Magdeburg, Cepartec)Goniva, C, Katterfeld, A, Kloss, C: „Simulation of dust emission and transport and chute wear“Proc. of 16. Fachtagung Schüttgutfördertechnik Magdeburg, Sept 2011Wear prediction (Finnie model)Fix mesh/gran/stressanalysis25 k particles, 4 processors:10 sec real time (200k steps)LB on: 1146 sec CPU timeLB off: 1514 sec CPU timeChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 23

IV. Features and Coding Concepts

Dynamic Load-BalancingWhat is Load-BalancingHow to distribute load between processors?Without dynamic load balancing:With dynamic load balancing:Process 0 Process 0Process 1 Process 1advancing simulation timeadvancing simulation timeChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 25

Dynamic Load-BalancingSilo Test CaseStrategy: “Simple” LB based on number of owned atomsLB offLB onSpeed-upChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 26

Coding ConceptsStrengths and Weak Points for LAMMPSStrengths of coding in LAMMPS• Code easy to read and understandd• Code easy to modify• Good encapsulation for many modelling tasks(fixes, computes, pair styles)But:• Have to write lots of lo-level code (error prone), some of which is redundantChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 27

Abstraction Layer I: Per-Particle PropertiesMotivationWe need a lot of different per-particle properties for our models• Temperature, heat flux, heat source (e.g. from chemical reactions)• Moisture content t (water in pores), liquid id surface film heightht• Particle Reynolds number• Drag force, Magnus force, Saffman force,... exerted by surrounding fluid• Surface dust content, dust emitted to fluid• Energy stored in elastic deformation, energy dissipated by contacts...• .... Need dozens of new per-particle properties for our modelsChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 28

Abstraction Layer I: Per-Particle PropertiesMotivationWhy not store new properties in Atom class?• Error prone: Have to implement low-level code via a new AtomVec class(pack/unpack, interproc exchange, restart...)• Have to change many classes to make new property fully functional(at least Atom, AtomVec, Dump, Set, ComputePropertyAtom, FixAdapt)• This is a recurring task, so we can add an abstraction layerSolution: Store it in a new fix property/peratom class• Implement low-level l l code only once• Look-up mechanism via Modify class so each other class can use it• Every model (fix) can now request to store a new per-particle particle propertyLazy storage: only store a property if any of the models really need it• Need to touch only one class for new property(dump, set... implemented only once for fix property/peratom)t Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 29

Abstraction Layer I: Per-Particle PropertiesUsage//register dragforceif(!dragforce){char* fixarg[11];fixarg[0]="dragforce";fixarg[1]="all";fixarg[2]="property/peratom";fixarg[3]="dragforce";fixarg[4]="vector";fixarg[5]="no";fixarg[6]="yes";fixarg[7]="no";fixarg[8]="0.";fixarg[9]="0.";fixarg[10]="0.";dragforce = modify->add_fix_property_peratom(11,fixarg);}Type of fixChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 30

Abstraction Layer I: Per-Particle PropertiesUsage//register dragforceif(!dragforce){char* fixarg[11];fixarg[0]="dragforce";fixarg[1]="all";fixarg[2]="property/peratom";fixarg[3]="dragforce";fixarg[4]="vector";fixarg[5]="no";fixarg[6]="yes";fixarg[7]="no";fixarg[8]="0.";fixarg[9]="0.";fixarg[10]="0.";dragforce = modify->add_fix_property_peratom(11,fixarg);}Name of propertyChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 31

Abstraction Layer I: Per-Particle PropertiesUsage//register dragforceif(!dragforce){char* fixarg[11];fixarg[0]="dragforce";fixarg[1]="all";fixarg[2]="property/peratom";fixarg[3]="dragforce";fixarg[4]="vector";fixarg[5]="no";fixarg[6]="yes";fixarg[7]="no";fixarg[8]="0.";fixarg[9]="0.";fixarg[10]="0.";dragforce = modify->add_fix_property_peratom(11,fixarg);}Type and size of dataChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 32

Abstraction Layer I: Per-Particle PropertiesUsage//register dragforceif(!dragforce){char* fixarg[11];fixarg[0]="dragforce";fixarg[1]="all";fixarg[2]="property/peratom";fixarg[3]="dragforce";fixarg[4]="vector";fixarg[5]="no";fixarg[6]="yes";fixarg[7]="no";fixarg[8]="0.";}fixarg[9]="0.";fixarg[10]="0.";dragforce = modify->add_fix_property_peratom(11,fixarg);Settings for forward comm,reverse comm, restartChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 33

Abstraction Layer II: “Transport Equations”MotivationThere is another recurring task: Solving an ODE for each particlefor a quantity, based on a flux and a sourceEg E.g. solveforheat transfer, moisture, evolution of surface dust...mpcp•Q pi −pjdTdtp,ipi••− +Q pi , source14243heat generationheat conduction due to sources,bycontacts e.g.reactions= ∑Q pi pjcontacts i − j14243 44 kpikpj/= Δk + kpj( A ) 1 2contact , i − jTpi−pjSolution: A new fix transportequation/scalar•Every model can request a scalar transport equation to be solved• The transportequation makes use of multiple fixes of type property/peratomChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 34

Abstraction Layer II: “Transport Equations”MotivationThere is another recurring task: Solving an ODE for each particlefor a quantity, based on a flux and a sourceEg E.g. solveforheat transfer, moisture, evolution of surface dust...mpcp•Q pi −pjdTdtp,ipi••− +Q pi , source14243heat generationheat conduction due to sources,bycontacts e.g.reactions= ∑Q pi pjcontacts i − j14243 44 kpikpj/= Δk + kpj( A ) 1 2contact , i − jTpi−pjSolution: A new fix transportequation/scalar•Every model can request a scalar transport equation to be solved• The transportequation makes use of multiple fixes of type property/peratomChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 35

V. Coupling to Continuum Methods

Coupling to Continuum MethodsMotivationMany problems are better modeled using a continuum approachOur main interest t lies in fluid mechanicsWeapon of choice: OpenFOAM®, www.openfoam.comOpen Field Operation and ManipulationLeader in OpenSource CFD, market share Germany ~30%, US ??• Initiated by Henry Weller and Hrvoje Jasak at Imperial in the 90s• Constant development by universities and companies,~ 2.000.000 LOC according to Jasak (end of 2009)• Capable of doing Finite Volume, Finite Element, Particle Methods(Lagrangian tracking, DSMC, MD), Electromagnetics, FinancialChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 37

Coupling to Continuum MethodsHow does OpenFOAM (R) work?fvVectorMatrix UEqn();solve();fvm::ddt(U) + fvm::div(phi, U) - fvm::laplacian(nu, U)UEqn == -fvc::grad(p)=OpenFOAM® is using the paradigm of object-orientedprogramming to make code to solve PDEs easily readableChristian Doppler Laboratory on Particulate Flow Modellingwww.particulate-flow.at39

Coupling to Continuum MethodsMotivationMotivation for using OpenFOAM® is manifold:•for postprocessing of DEM/MD simulations• for simulation of coupled fluid-granular l flow (FV method)• can use the full model portfolio of OF (FE, electrostatics,...)particle dataOpenFoam® SolverLIGGGHTSparticle data feedbackMore information and downloads: www.cfdem.comcomChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 40

Coupling to Continuum MethodsMotivationPost-processing of DEM/MD data• Able to transfer arbitrary particle properties to OpenFOAM fields• Can use arbitrary unstructured meshes• Example: Evaluating voidage, which h is an important t process parameterin multiphase flow reactorsChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 41

Coupling to Continuum MethodsMotivationSimulation of Coupled Fluid-Granular Flow: CFD-DEM Method• Able to transfer arbitrary particle properties to OpenFOAM fields• Can use arbitrary unstructured meshes• Able to perform resolved CFD-DEM DEM and unresolved CFD-DEMDEM-(f1 +f2)αf,1αf,2= +-(f3 + f4)ff1f2f4f3CFD with adapted voidfraction andmomentum sourcesDEM with forcecoming from CFD= +Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 42

Coupling to Continuum MethodsUnresolved CFD-DEMDEMTheoretical background:Navier-Stokes equations for the fluid in presence of a granular phase∂α∂f∂ tρf+ ∇ ⋅( α ρ uf)f∂ tf( α ρ u )f+ ∇ ⋅ff= 0( α ρ ufuf) = −α∇p− ( uf−us) + ∇ ⋅( α τ) + α ρ gfffK fsfffα f fluid volume fractionu f fluid velocityτ, p stress tensor, pressureρ f fluid densityK fs fluid solid momentum exchange termcomprises drag force, Magnus and Saffman force, virtual mass force,...Christian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 43

Coupling to Continuum MethodsUnresolved CFD-DEMDEMTheoretical background:Example for fluid solid momentum exchange K fs :Di Felice (1994): “The voidage function for fluid‐particle interaction systems.” Int. J. of Multiphase Flow,Vol. 20, p153‐159159FCdd=21 pπ 1ρ (f−p) f−p− χfCdαf,2uu⎛4.8= 0.63+⎜⎝Re⎞⎟⎠⎡χ =3.7 −0.65exp−⎢ ⎢ ⎣p2u,ud42( 1.5 − log Re ) ⎤⎥210 pK fs=α fV cellBetter models available are based on Lattice Boltzmann simulation (e.g. Koch and Hill)⎥⎥⎦⋅⋅∑iu −F dfu pChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 44

Application ExampleSoil Sampling with “Mole”Transfer data to CFDFetch DEM particle dataadditionalmodelsCalculate particle bondsCalculate object motionCalculate particle motionCalculate flowCalculate moving meshCalculate heat transferadditionalmodelsFetch CFD dataCalculate volume fractionand momentum couplingstartTransfer data to DEMChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 46

Application ExampleSoil Sampling with “Mole”moving mesh application: coupling of Temperature and object motionuniform lattice bonds break mole movesheated moleuniform lattice bonds resist mole fixedcold moletimeChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 47

Industrial ApplicationReduction of Dust Emission in a Transfer ChuteWork with Andre Katterfeld (Univ. Magdeburg, Cepartec)Goniva, C, Katterfeld, A, Kloss, C: „Simulation of dust emission and transport and chute wear“Proc. of 16. Fachtagung Schüttgutfördertechnik Magdeburg, Sept 2011Original GeometryChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 48

Industrial ApplicationReduction of Dust Emission in a Transfer ChuteWork with Andre Katterfeld (Univ. Magdeburg, Cepartec)Goniva, C, Katterfeld, A, Kloss, C: „Simulation of dust emission and transport and chute wear“Proc. of 16. Fachtagung Schüttgutfördertechnik Magdeburg, Sept 2011Optimized GeometryChristian Doppler Laboratory on Particulate Flow Modelling www.particulate-flow.at 49

Thank you for your attention!Questions?www.cfdem.com | www.particulate-flow.at