Gluster Internals- Jeff Darcy
Home |
---|
See also:
- Understanding Various Translators
- Understanding RPC Internals
- Current GeoRep
- UFO and Swift Integration
Contents[edit | edit source]
- 1 Getting Started: Jeff Darcy Tutorials
- 2 Translator API
- 3 Gluster CLI
- 4 Configuration and Vol Files
Getting Started: Jeff Darcy Tutorials[edit | edit source]
An excellent set of tutorials can be found on hekafs.org blog
- Translator 101 Lesson 1: Setting the Stage * Translator 101 Lesson 2: init, fini, and private context * Translator 101 Lesson 3: This Time For Real * Translator 101 Lesson 4: Debugging a Translator
Translator API[edit | edit source]
External APIs[edit | edit source]
GlusterFS uses Extended Attributes as a mechanism for external interaction with translators. Unlike the static values of a transitional Posix file or directory access, Glusterfs xattr values can be dynamically generated and setxattr can be thought of as a form of method invocation. The HekaFs Blog contains a good discussion of GlusterFS Extended Attributes.
Glusterfs extended attributes include:
- trusted.glusterfs.afr
- trusted.glusterfs.afr.data-pending
- trusted.glusterfs.afr.metadata-pending
- trusted.glusterfs.afr.entry-pending
- client
- trusted.glusterfs.client-connect
- trusted.glusterfs.client-disconnect
- trusted.glusterfs.clientlk-dump
- trusted.glusterfs.createtime
- trusted.glusterfs.dht
- trusted.glusterfs.dht.linkto
- trusted.glusterfs.node-uuid
- trusted.glusterfs.pathinfo
- trusted.glusterfs.pump
- trusted.glusterfs.pump-path
- trusted.glusterfs.pump-source-complete
- trusted.glusterfs.pump-sink.complete
- trusted.glusterfs.quota
- trusted.glusterfs.quota.dirty
- trusted.glusterfs.quotq.size
- trusted.glusterfs.quota.*.contri
- trusted.glusterfs-quota-du
- trusted.glusterfs.set-log-level
- trusted.glusterfs.*.set-log-level
- trusted.glusterfs.fuse.set-log-level
- trusted.glusterfs.*.set-log-level
- trusted.glusterfs.syslog
- trusted.glusterfs.test
- trusted.glusterfs.version
- volume
- trusted.glusterfs.volume-id
- trusted.glusterfs.volume-mark
- trusted.glusterfs.xtime or system.glusterfs.xtime
- trusted.glusterfs.*.xtime
Other extended attributes include:
- trusted.afr.createtime
- trusted.afr.version
- trusted.afr.volume
- trusted.distribute.linkinfo
- trusted.gfid
- trusted.limit.list
- trusted.quota*
- trusted.testing
- trusted.xattr-test
Internal APIs[edit | edit source]
Gluster translator APIs are contained in libglusterfs
. Useful documentation of the various APIs can be found on Hekafs.org.
APIs and support functions includes:
- Dispatch Tables and Default Functions
- STACK_WIND and STACK_UNWIND
- Per Request Context
- Inode and File Descriptor Context
- Dictionaries and Translator Options
- logging
- Child Enumeration and Fan Out
Gluster file and directory operations closely model those of FUSE. The FUSE sourceforge site contains useful information on definition of many of these operations.
Gluster CLI[edit | edit source]
Hooks Mechanism[edit | edit source]
Hooks Mechanisms:
Hooks — from: glusterfs / xlators / mgmt / glusterd / src / glusterd-hooks.c
#define EMPTY "" char glusterd_hook_dirnames[GD_OP_MAX][256] = { [GD_OP_NONE] = EMPTY, [GD_OP_CREATE_VOLUME] = "create", [GD_OP_START_BRICK] = EMPTY, [GD_OP_STOP_BRICK] = EMPTY, [GD_OP_DELETE_VOLUME] = "delete", [GD_OP_START_VOLUME] = "start", [GD_OP_STOP_VOLUME] = "stop", [GD_OP_DEFRAG_VOLUME] = EMPTY, [GD_OP_ADD_BRICK] = "add-brick", [GD_OP_REMOVE_BRICK] = "remove-brick", [GD_OP_REPLACE_BRICK] = EMPTY, [GD_OP_SET_VOLUME] = "set", [GD_OP_RESET_VOLUME] = EMPTY, [GD_OP_SYNC_VOLUME] = EMPTY, [GD_OP_LOG_ROTATE] = EMPTY, [GD_OP_GSYNC_SET] = EMPTY, [GD_OP_PROFILE_VOLUME] = EMPTY, [GD_OP_QUOTA] = EMPTY, [GD_OP_STATUS_VOLUME] = EMPTY, [GD_OP_REBALANCE] = EMPTY, [GD_OP_HEAL_VOLUME] = EMPTY, [GD_OP_STATEDUMP_VOLUME] = EMPTY, [GD_OP_LIST_VOLUME] = EMPTY, [GD_OP_CLEARLOCKS_VOLUME] = EMPTY, [GD_OP_DEFRAG_BRICK_VOLUME] = EMPTY, };
Configuration and Vol Files[edit | edit source]
Sample Glusterd Vol File[edit | edit source]
Used to configure the glusterd management agent. Can be found at `/etc/glusterfs/glusterd.vol `
volume management type mgmt/glusterd option working-directory /var/lib/glusterd option transport-type socket,rdma option transport.socket.keepalive-time 10 option transport.socket.keepalive-interval 2 option transport.socket.read-fail-log off end-volume
Sample Client-side Vol File[edit | edit source]
Used to configure each volume available to a client. Files can be found in /var/lib/glusterd/vols/vol_name/
subdirectory
volume my_vol-client-0 type protocol/client option remote-host 192.168.122.125 option remote-subvolume /glusterdata/brick125 option transport-type tcp end-volume volume my_vol-client-1 type protocol/client option remote-host 192.168.122.236 option remote-subvolume /glusterdata/brick236 option transport-type tcp end-volume volume my_vol-dht type cluster/distribute subvolumes my_vol-client-0 my_vol-client-1 end-volume volume my_vol-write-behind type performance/write-behind subvolumes my_vol-dht end-volume volume my_vol-read-ahead type performance/read-ahead subvolumes my_vol-write-behind end-volume volume my_vol-io-cache type performance/io-cache subvolumes my_vol-read-ahead end-volume volume my_vol-quick-read type performance/quick-read subvolumes my_vol-io-cache end-volume volume my_vol-md-cache type performance/md-cache subvolumes my_vol-quick-read end-volume volume my_vol type debug/io-stats option latency-measurement off option count-fop-hits off subvolumes my_vol-md-cache end-volume
There is often a trusted version of the vol file located in the same directory. Below is an example of the additional information in protocol/client entry for the trusted variant.
volume my_vol-client-0 type protocol/client option remote-host 192.168.122.125 option remote-subvolume /glusterdata/brick125 option transport-type tcp option username 79431724-6ea3-482f-8d2e-7654d8934d83 option password be6b8ca2-89fb-4223-b521-d976e3c5f6cd end-volume
Sample Server-side Vol File[edit | edit source]
Used to configure each brick in the trusted storage pool. Files can be found in ` /var/lib/glusterd/vols/vol_name/` subdirectory
volume my_vol-posix type storage/posix option directory /glusterdata/brick125 option volume-id 5e2d14d7-8352-46ca-8b7f-b965c277871f end-volume volume my_vol-access-control type features/access-control subvolumes my_vol-posix end-volume volume my_vol-locks type features/locks subvolumes my_vol-access-control end-volume volume my_vol-io-threads type performance/io-threads subvolumes my_vol-locks end-volume volume my_vol-index type features/index option index-base /glusterdata/brick125/.glusterfs/indices subvolumes my_vol-io-threads end-volume volume my_vol-marker type features/marker option volume-uuid 5e2d14d7-8352-46ca-8b7f-b965c277871f option timestamp-file /var/lib/glusterd/vols/my_vol/marker.tstamp option xtime off option quota off subvolumes my_vol-index end-volume volume /glusterdata/brick125 type debug/io-stats option latency-measurement off option count-fop-hits off subvolumes my_vol-marker end-volume volume my_vol-server type protocol/server option transport-type tcp option auth.login./glusterdata/brick125.allow 79431724-6ea3-482f-8d2e-7654d8934d83 option auth.login.79431724-6ea3-482f-8d2e-7654d8934d83.password be6b8ca2-89fb-4223-b521-d976e3c5f6cd option auth.addr./glusterdata/brick125.allow * subvolumes /glusterdata/brick125 end-volume
Brick file[edit | edit source]
Additional information on each brick can be found in /var/lib/glusterd/vols/vol_name/bricks
subdirectory
hostname=192.168.122.125 path=/glusterdata/brick125 listen-port=24010 rdma.listen-port=0 decommissioned=0
Home |
---|