User Tools

Site Tools


data_management:irods

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
data_management:irods [2019/08/19 10:17]
jsteinka [Navigation]
data_management:irods [2020/01/30 12:36] (current)
jsteinka [Metadata: ''imeta'']
Line 1: Line 1:
 +<WRAP center round important 60%>
 +Please note, that this service is still in its beta-phase. Do not hesitate to [[training_and_outreach:​ticket_system|contact us]] in order to discuss your data management plan.
 +</​WRAP>​
 +
 ====== Archiving with iRODS ====== ====== Archiving with iRODS ======
  
Line 5: Line 9:
   *  files are (//data objects//).   *  files are (//data objects//).
  
-All commands from the iRODS command line tools start with an '//​i//'​.+All commands from the iRODS command line tools start with an '//​i//' ​and they are installed on the mogon login nodes. 
 + 
 +iRODS uses so called '//​Resources//'​ to archive the //​collections//​ and //data objects//. The //​resources//​ are organized hierarchically. The root is a //​replication resource//, where other //​resources//​ are added as children. Currently there is a //compound resource// consisting itself of a cache (unix filesystem) and a universal mass storage system (here: TSM) as archive. The cache has a size of 8TB, once it is fills up, the oldest //data objects// will be deleted on the cache. If required, they will be fetched back from the archive. 
 + 
 +<​code>​ 
 +replResc:​replication 
 +├── cephfsResc:​unixfilesystem 
 +└── compResc:​compound 
 +    ├── netappResc:​unixfilesystem 
 +    └── tsmResc:​univmss 
 +</​code>​
  
 On request it is still possible to apply for a [[archiving:​tsm|TSM account]], to store the data on tapes. On request it is still possible to apply for a [[archiving:​tsm|TSM account]], to store the data on tapes.
Line 18: Line 32:
  
 If a directory, resp. //​collection//,​ consists of many small files, those files should be [[archiving:​preparation|compressed]]. iRODS works best for files > 5GB. If a directory, resp. //​collection//,​ consists of many small files, those files should be [[archiving:​preparation|compressed]]. iRODS works best for files > 5GB.
 +{{ :​data_management:​test-large-files.png?​direct&​400 | iRODS upload benchmark with one file of varying size.}} 
 +{{ :​data_management:​test-small-files.png?​direct&​400 | iRODS upload benchmark with multiple tiny files.}}
 ===== iRODS Account ===== ===== iRODS Account =====
  
Line 27: Line 42:
  
 Collections in the home folder of individual user will be deleted once the account gets deleted. Only the // /​zdv/​project/<​PROJECT NAME> // collections will be archived for an appropriate period (default: 10 years, as suggested by the [[https://​www.dfg.de/​foerderung/​grundlagen_rahmenbedingungen/​gwp/​|DFG,​ Leitlinie 17]]). Collections in the home folder of individual user will be deleted once the account gets deleted. Only the // /​zdv/​project/<​PROJECT NAME> // collections will be archived for an appropriate period (default: 10 years, as suggested by the [[https://​www.dfg.de/​foerderung/​grundlagen_rahmenbedingungen/​gwp/​|DFG,​ Leitlinie 17]]).
 +
 +The data saved to the above project collection is owned by the group. This means after a user leaves the project, the data can still be accessed by the other group members, as long as the ACLs are not modified.
 </​WRAP>​ </​WRAP>​
  
Line 51: Line 68:
 **Security warning** **Security warning**
  
-If you initiate the command ''​iinit''​ it might happen that an additional file '//​.irodsA//'​ is also in the folder '//​.irods//'​. Please remove this file! It contains your entered ​password in a decryptable form.+If you initiate the command ''​iinit''​ it might happen that an additional file '//​.irodsA//'​ is also in the folder '//​.irods//'​. Please remove this file! It contains your password in a decryptable form.
 </​WRAP>​ </​WRAP>​
  
Line 72: Line 89:
 Accessible folders: Accessible folders:
   * ''/​zdv/​home/​${USER}''​ private directory   * ''/​zdv/​home/​${USER}''​ private directory
-  * ''/​zdv/​trash/​${USER}''​ private trash bin 
   * ''/​zdv/​project/<​PROJECT NAME>''​ project directory   * ''/​zdv/​project/<​PROJECT NAME>''​ project directory
   * ''/​zdv/​home/​public''​ every registered user can read/​write/​delete   * ''/​zdv/​home/​public''​ every registered user can read/​write/​delete
 +  * ''/​zdv/​trash/​home/​${USER}''​ private trash bin
 +
 +
 +<WRAP center round info 80%>
 +Only the project directory are meant for permanent archiving. We are working on a solution to prevent archiving to the **volatile** iRODS homes.
 +</​WRAP>​
  
 ==== Archiving ==== ==== Archiving ====
Line 84: Line 106:
 | ''​ichksum''​ | -r <​obj%%|%%coll>​ | Compute and store checksums (-r: recursive) ​                   | | ''​ichksum''​ | -r <​obj%%|%%coll>​ | Compute and store checksums (-r: recursive) ​                   |
  
-The checksum is calculated server side and we highly recommend to switch it on immediately on upload. Nevertheless,​ You can also do it later on with ''​ichksum -K <​filename>''​. It creates a checksum equivalent to the command ''​sha256sum <local filename>​ | cut -d " " -f 1 | xxd -r -p | base64'',​ which you can compare to ensure data integrity. The checksums can be queried with ''​ils -L''​ and ''​ichksum''​.+The checksum is calculated server side and we highly recommend to switch it on immediately on upload. Nevertheless,​ You can also do it later on with ''​ichksum -K <​filename>''​. It creates a checksum equivalent to the command ''​sha256sum <local filename>​ | cut -d " " -f 1 | xxd -r -p | base64'',​ which you can compare to ensure data integrity. The checksums can be queried with ''​ils -L''​ and ''​ichksum''​. However, if you don't do it on upload with ''​iput -k'',​ there will be no checksum for the TSM ressource.
  
 As mentioned above, several small files should be bundled. Nevertheless,​ you can still extract an uploaded tar archive on the server and index all contained files with the command ''​ibun''​. Please read its man page for further details (''​man ibun''​). As mentioned above, several small files should be bundled. Nevertheless,​ you can still extract an uploaded tar archive on the server and index all contained files with the command ''​ibun''​. Please read its man page for further details (''​man ibun''​).
Line 106: Line 128:
 | -r        | recursive ​                     | | -r        | recursive ​                     |
 | -f        | overwrite local existing files | | -f        | overwrite local existing files |
 +
 +=== Example ===
 +1. Get your kerberos ticket and print some information about your iRODS account
 +{{ :​data_management:​irods_01_kinit.png?​600 |}}
 +
 +2. Navigation
 +{{ :​data_management:​irods_02_navigation.png?​600 |}}
 +
 +3. Archiving, retrieving files and simple information
 +{{ :​data_management:​irods_03_archiving_files.png?​600 |}}
 +
 +4. Checksum
 +{{ :​data_management:​irods_04_checksum.png?​800 |}}
  
 ==== Metadata: ''​imeta''​ ==== ==== Metadata: ''​imeta''​ ====
Line 129: Line 164:
  
  
-The following ​two commands upload the filesystem table and query the metadata:+The following ​command lists the metadata ​automatically associated with the previously upladed file ''​hello_world.txt''​:
 <code bash> <code bash>
-iput <​path/​filename>​ +imeta ls -d hello_world.txt
-imeta ls -d <​filename>​+
 </​code>​ </​code>​
  
Line 138: Line 172:
  
 <​code>​ <​code>​
-AVUs defined for dataObj ​<​filename>​:+AVUs defined for dataObj ​hello_world.txt:
 attribute: AccessRights attribute: AccessRights
 value: closed value: closed
Line 144: Line 178:
 ---- ----
 attribute: Creator attribute: Creator
-value: ​<you>+value: ​Steinkamp, J.
 units: ​ units: ​
 ---- ----
 attribute: Date attribute: Date
-value: ​1562677318+value: ​1566206896
 units: ​ units: ​
 ---- ----
 attribute: ExpiryDate attribute: ExpiryDate
-value: ​1878901318+value: ​1882430896
 units: ​ units: ​
 ---- ----
Line 166: Line 200:
 value: Johannes Gutenberg-University value: Johannes Gutenberg-University
 units: ​ units: ​
 +[user@login01 ~]$ 
 </​code>​ </​code>​
  
 +You can now add a title, which is not created automatically:​
 <​code>​ <​code>​
-imeta set -d <​filename> ​Title "​Archive of experimental szstem from '​$(date)'"​+imeta set -d hello_world.txt ​Title "​Archive of experimental szstem from '​$(date)'"​
 </​code>​ </​code>​
  
-If you query the Attribute '​Table'​ with ''​imeta ls -d <​filename> ​Title''​ you get:+If you query the Attribute '​Table'​ with ''​imeta ls -d hello_world.txt ​Title''​ you get:
 <​code>​ <​code>​
-AVUs defined for dataObj ​fstab:+AVUs defined for dataObj ​hello_world.txt:
 attribute: Title attribute: Title
-value: Archive of experimental szstem from '<​somedate>​'+value: Archive of experimental szstem from 'Mon Aug 19 11:39:48 CEST 2019'
 units: ​ units: ​
 </​code>​ </​code>​
Line 191: Line 227:
  
 <code shell> <code shell>
-imeta mod -d <​filename> ​Title "​Archive of experimental szstem from '<​somedate>​'"​ v:"​Archive of experimental system from '<​somedate>​'"​+imeta mod -d hello_world.txt ​Title "​Archive of experimental szstem from 'Mon Aug 19 11:39:48 CEST 2019'"​ v:"​Archive of experimental system from 'Mon Aug 19 11:39:48 CEST 2019'"​
 </​code>​ </​code>​
  
Line 201: Line 237:
 As you could see above, we generate as many metadata attributes as possible automatically,​ to hopefully simplify your life. Nevertheless,​ you can adjust and extend them to your needs. As you could see above, we generate as many metadata attributes as possible automatically,​ to hopefully simplify your life. Nevertheless,​ you can adjust and extend them to your needs.
  
- +  ​Set automatically:​ 
-  ​**Title** free text (//user input needed//) +    * **Creator** full user name 
-  * **Creator** full user name (//created automatically//​) +    * **Publisher** "​Johannes Gutenberg-University"​ 
-  * **Publisher** "​Johannes Gutenberg-University" ​(//created automatically//​) +    * **Location** "​Mainz,​ Germany"​ 
-  * **Location** "​Mainz,​ Germany" ​(//created automatically//​) +    * **Date** Unix timestamp 
-  * **Date** Unix timestamp ​(//created automatically//​) +    * **ExpiryDate** Date + 10 years 
-  * **ExpiryDate** Date + 10 years (//created automatically//) +    * **protected** ​(//default: "​false"​//) 
-  * **Type** audio, data set, image, source code, ... (//user input needed//) +  ​* User input required: 
-  * **Format** simply the file format (e.g. output from *file* command) (//user input needed//+    * **Title** free text 
-  * **AccessRights** "​closed",​ "​restricted",​ "​embargoed",​ "​open"​ (//default: "​closed"//​) +    * **Description** text 
-    * **AccessConditions** if AccessRights is "​resticted"​ (not yet) +    ​* **Type** audio, data set, image, source code, ... 
-    * **EmbargoDate** if AccessRights is "​embargoed"​ (not yet+    * **Format** simply the file format (e.g. output from *file* command) 
-  * **protected** (//default: "​false"//​)+    * **AccessRights** "​closed",​ "​restricted",​ "​embargoed",​ "​open"​ (//default: "​closed"//​) 
 +      * **AccessConditions** if AccessRights is "​resticted"​ (not yet) 
 +      * **EmbargoDate** if AccessRights is "​embargoed"​ (not yet)
  
 <WRAP center round info 80%> <WRAP center round info 80%>
Line 225: Line 263:
   * **Identifier** (provided by ZDV/UB, only if attribute "​protected"​ is set; not yet)   * **Identifier** (provided by ZDV/UB, only if attribute "​protected"​ is set; not yet)
   * **License** The license for reuse. Recommended:​ GPL for code, CC0 for data sets, otherwise CC-BY   * **License** The license for reuse. Recommended:​ GPL for code, CC0 for data sets, otherwise CC-BY
-  * **Subject** any keywords+  * **Keywords** any keywords
  
 === Additional Recommended Attributes === === Additional Recommended Attributes ===
Line 231: Line 269:
   * **Contributor** co-authors   * **Contributor** co-authors
   * **Reference** publication references   * **Reference** publication references
-  * **Description** free test 
-  * **Abstract** free text 
  
  
Line 241: Line 277:
     * [[https://​www.ddialliance.org/​Data Documentation Initiative]]     * [[https://​www.ddialliance.org/​Data Documentation Initiative]]
     * [[https://​www.radar-service.eu/​radar-schema|RADAR]]     * [[https://​www.radar-service.eu/​radar-schema|RADAR]]
 +    * [[https://​schema.org/​|schema.org]]
   * subject specific   * subject specific
     * [[http://​www.dcc.ac.uk/​resources/​metadata-standards|Digital Curation Centre]]     * [[http://​www.dcc.ac.uk/​resources/​metadata-standards|Digital Curation Centre]]
Line 246: Line 283:
 ==== Searching ==== ==== Searching ====
  
 +=== for filenames: ''​ilocate''​ ===
  
-  * ''​ilocate -t "​%fstab%"''​ search for filename (pattern) +<code bash>
-  * ''​imeta qu -C Name like "​%fstab%"''​ search for metadata+
  
-==== Publishing ====+[user@login01 ~]$ ilocate -t "​hello_world.txt"​  
 +/​zdv/​home/​user/​hello_world.txt 
 +/​zdv/​home/​public/​hello_world.txt 
 +</​code>​
  
-For public access a **ticket** needs to be created ​for collections or data objectsFor example, if you use the above uploaded ​''​fstab'' ​file (here for my personal home directorywhich might not work in the future any more, see above why).+=== for metadata: ''​imeta qu''​ === 
 + 
 +You must know, if you want to search ​for data object (''​-d''​) or a collection (''​-C''​)And you can use SQL wildcards (''​%''​)if you don't know the exact pattern you are looking for. The wildcard pattern matching is also applicable for ''​ilocate''​.
  
 <code bash> <code bash>
-iticket create read fstab+[user@login01 ~]$ imeta qu -d Title like "​Archive%"​ 
 +collection: /​zdv/​home/​user 
 +dataObj: hello_world.txt
 </​code>​ </​code>​
-returns:+ 
 +=== via database query''​iquest''​ === 
 + 
 +For this complex syntax consult the [[https://​docs.irods.org/​4.2.6/​icommands/​user/#​iquest|Online help]]. 
 + 
 +==== Publishing ==== 
 + 
 +For public access a **ticket** needs to be created for collections or data objects. For example, if you use the above uploaded ''​hello_world.txt''​ file again. 
 <code bash> <code bash>
-ticket:GGkUTXdJpfK7VPi+[user@login01 ~]$ iticket create read hello_world.txt 
 +ticket:ACR2RKDyuZMBRmb
 </​code>​ </​code>​
  
-Querying ​the metadata via the provided REST-API ​returns a JSON stringThis can be viewed in the browser or as done here using curl:+With this ticket and the path everybody can query information and the content of collections and data objects vi a provided REST-APIJSON strings are returned for valid URLs. 
 + 
 +=== General information about data objects ===
  
 <code bash> <code bash>
-curl https://​irods-test.zdv.uni-mainz.de/​irods-rest/​rest/​dataObject/​zdv/​home/​jsteinka/​fstab/​metadata?ticket=GGkUTXdJpfK7VPi+[user@login01 ~]$ curl https://​irods-web.zdv.uni-mainz.de/​irods-rest/​rest/​dataObject/​zdv/​home/​jsteinka/​hello_world.txt?ticket=ACR2RKDyuZMBRmb
 </​code>​ </​code>​
-returns: 
 <code JavaScript>​ <code JavaScript>​
-{"metadataEntries": ​+{"id":1808764
-  {"​count":​1+ "​collectionId":24346
-   ​"lastResult":true+ "​dataName":"​hello_world.txt", 
-   ​"totalRecords":0, + "​collectionName":"​/​zdv/​home/​jsteinka", 
-   "attribute":"​protected", + "​dataReplicationNumber":0, 
-   ​"value":"​false", + "​dataVersion":0, 
-   ​"unit":""​},​  + "​dataTypeName":"​generic", 
-  {"​count":​2,​ + "​dataSize":24
-   "​lastResult":​true,​ + "​resourceGroupName":"",​ 
-   "​totalRecords":0, + "​resourceName":"​netappResc", 
-   "​attribute":"​AccessRights",​ + "​dataPath":"​/​fsapp/​iRODS/​Vault/​home/​jsteinka/​hello_world.txt", 
-   "​value":"​closed",​ + "​dataOwnerName":"​jsteinka", 
-   "​unit":""​},​ + "​dataOwnerZone":"​zdv", 
-  {"​count":​3,​ + "​replicationStatus":"​1", 
-   "​lastResult":​true,​ + "​dataStatus":"",​ 
-   "totalRecords":0, + "​checksum":"​sha2:XPdR4XQP49lWUGEfPJz0Jo+kmkndGxz6rCQUzCqHteA=", 
-   ​"attribute":"​Publisher", + "​expiry":"​00000000000", 
-   ​"value":"​Johannes Gutenberg-University"​+ "​dataMapId":0, 
-   ​"unit":""​}+ "​comments":"",​ 
-  {"count":4, + "​createdAt":1566206868000
-   "lastResult":true+ "​updatedAt":1566206868000
-   ​"totalRecords":0, + "​specColType":"​NORMAL", 
-   "attribute":"​Location", + "​objectPath":""​ 
-   ​"value":"​Mainz, Germany", +}
-   ​"unit":""​}+
-  {"count":5, +
-   "lastResult":true+
-   ​"totalRecords":0, +
-   "attribute":"​Creator", +
-   ​"value":"​Steinkamp, Jörg",​ +
-   "​unit"​:""}+
-  {"count":6, +
-   "lastResult":true+
-   ​"totalRecords":0, +
-   ​"attribute":"​Title", +
-   ​"value":"​Filesystem Table of '​login01.mogon'"​+
-   ​"unit":""​}+
-  {"count":7, +
-   "lastResult":true+
-   ​"totalRecords":0, +
-   "attribute":"​Date",​ +
-   "​value":"​1562679909",​ +
-   "​unit":""​},​ +
-  {"​count":​8,​ +
-   "​lastResult":​true,​ +
-   "​totalRecords":​0,​ +
-   "​attribute":"​ExpiryDate",​ +
-   "​value":"​1878903909",​ +
-   "​unit":""​}],​ +
-"​objectType":"​DATA_OBJECT",​ +
-"​uniqueNameString":"/​zdv/​home/​jsteinka/​fstab"​}+
 </​code>​ </​code>​
  
 +=== Querying the metadata ===
  
-  * Retrieve the metadata of a collection: 
 <code bash> <code bash>
-curl https://​irods-test.zdv.uni-mainz.de/​irods-rest/​rest/​collection/​zdv/​home/​jsteinka/​test/​metadata?​ticket=AKR9iYWmfSU6niN +[user@login01 ~]$ curl https://​irods-web.zdv.uni-mainz.de/​irods-rest/​rest/​dataObject/​zdv/​home/​jsteinka/​hello_world.txt/​metadata?​ticket=ACR2RKDyuZMBRmb
-{"​metadataEntries":​[ +
-        {"​count":​1,​ +
-         "​lastResult":​true,​ +
-         "​totalRecords":​0,​ +
-         "​attribute":"​Name",​ +
-         "​value":"​This is a test project","​unit":""​}],​ +
-    "​objectType":"​COLLECTION",​ +
-    "​uniqueNameString":"​zdv/​home/​jsteinka/​test"​ +
-}+
 </​code>​ </​code>​
-  * Retrieve the metadata ​of a data object:+<code JavaScript>​ 
 +{"​metadataEntries":​ [ 
 +    {"​count":​1,​ 
 +     "​lastResult":​true,​ 
 +     "​totalRecords":​0,​ 
 +     "​attribute":"​AccessRights",​ 
 +     "​value":"​closed",​ 
 +     "​unit":""​},​ 
 +    {"​count":​2,​ 
 +     "​lastResult":​true,​ 
 +     "​totalRecords":​0,​ 
 +     "​attribute":"​Publisher",​ 
 +     "​value":"​Johannes Gutenberg-University",​ 
 +     "​unit":""​},​ 
 +    {"​count":​3,​ 
 +     "​lastResult":​true,​ 
 +     "​totalRecords":​0,​ 
 +     "​attribute":"​Location",​ 
 +     "​value":"​Mainz,​ Germany",​ 
 +     "​unit":""​},​ 
 +    {"​count":​4,​ 
 +     "​lastResult":​true,​ 
 +     "​totalRecords":​0,​ 
 +     "​attribute":"​protected",​ 
 +     "​value":"​false",​ 
 +     "​unit":""​},​ 
 +    {"​count":​5,​ 
 +     "​lastResult":​true,​ 
 +     "​totalRecords":​0,​ 
 +     "​attribute":"​Creator",​ 
 +     "​value":"​Steinkamp,​ J.", 
 +     "​unit":""​},​ 
 +    {"​count":​6,​ 
 +     "​lastResult":​true,​ 
 +     "​totalRecords":​0,​ 
 +     "​attribute":"​Date",​ 
 +     "​value":"​1566206896",​ 
 +     "​unit":""​},​ 
 +    {"​count":​7,​ 
 +     "​lastResult":​true,"​ 
 +     ​totalRecords":​0,​ 
 +     "​attribute":"​ExpiryDate",​ 
 +     "​value":"​1882430896",​ 
 +     "​unit":""​},​ 
 +    {"​count":​8,​ 
 +     "​lastResult":​true,​ 
 +     "​totalRecords":​0,​ 
 +     "​attribute":"​Title",​ 
 +     "​value":"​Archive ​of experimental system from 'Mon Aug 19 11:39:48 CEST 2019'",​ 
 +     "​unit":""​}],​ 
 +"​objectType":"​DATA_OBJECT","​uniqueNameString":"/​zdv/​home/​jsteinka/​hello_world.txt"​} 
 +</​code>​ 
 + 
 +=== Downloading data === 
 +The file content can be viewed with ''​curl''​ or downloaded with ''​wget''​.
 <code bash> <code bash>
-curl https://​irods-test.zdv.uni-mainz.de/​irods-rest/​rest/​dataObject/​zdv/​home/​jsteinka/​test/​garbage_small.zero/​metadata?ticket=qM3arKFlkBM0Ue5 +curl https://​irods-web.zdv.uni-mainz.de/​irods-rest/​rest/​fileContents/​zdv/​home/​jsteinka/​hello_world.txt?ticket=ACR2RKDyuZMBRmb
-{"​metadataEntries":​[ +
-        {"​count":​1,​ +
-         "​lastResult":​true,​ +
-         "​totalRecords":​0,​ +
-         "​attribute":"​Creator",​ +
-         "​value":"​Joerg Steinkamp",​ +
-         "​unit":""​}, ​  +
-        {"​count":​2,​ +
-         "​lastResult":​true,​ +
-         "​totalRecords":​0,​ +
-         "​attribute":"​Description",​ +
-         "​value":"​only zeroes from dd if=/​dev/​zero of=garbage_small-zero bs=1024 count=24",​ +
-         "​unit":""​}],​ +
-    "​objectType":"​DATA_OBJECT",​ +
-    "​uniqueNameString":"/​zdv/​home/​jsteinka/​test/​garbage_small.zero"​ +
-}+
 </​code>​ </​code>​
-  * Download ​the data object:+ 
 +=== Retrieve ​the metadata of a collection ===
 <code bash> <code bash>
-wget  ​https://​irods-test.zdv.uni-mainz.de/​irods-rest/​rest/​fileContents/zdv/home/jsteinka/test/​garbage_small.zero?ticket=qM3arKFlkBM0Ue5+curl https://​irods-web.zdv.uni-mainz.de/​irods-rest/​rest/​collection/zdv/home/public/helloCollection?ticket=mbyAAFGm7vhUdyM
 </​code>​ </​code>​
 +<code JavaScript>​
 +{
 + "​collectionId":​1808781,​
 + "​collectionName":"/​zdv/​home/​public/​helloCollection",​
 + "​objectPath":"",​
 + "​collectionParentName":"/​zdv/​home/​public/",​
 + "​collectionOwnerName":"​rods",​
 + "​collectionOwnerZone":"​zdv",​
 + "​collectionMapId":"​0",​
 + "​collectionInheritance":"",​
 + "​comments":"",​
 + "​info1":"",​
 + "​info2":"",​
 + "​createdAt":​1566213731000,​
 + "​modifiedAt":​1566213731000,​
 + "​specColType":"​NORMAL",​
 + "​children":​[]
 +}
 +</​code>​
 +
 +=== REST-API URL ===
 +
 +the URL for the REST-API consists of:
 +  * Server (https://​irods-web.zdv.uni-mainz.de/​irods-rest/​rest)
 +  * what (collection|dataObject|fileContents)
 +  * iRODS path
 +  * optionally '​metadata'​
 +  * Ticket string (?​ticket=1234567890)
  
 +For further information,​ please read the original [[https://​github.com/​DICE-UNC/​irods-rest|IRODS-REST documentation]]
 ===== Data Policy/​Recommendation ===== ===== Data Policy/​Recommendation =====
  
 The "​Creator"​ is the responsible person in the sense of the Urheberrechtsgesetz,​ taking care that reusing of third party data is legal and in the sense of the DSGVO, that personal data is handled correctly. Even if the "​Creator"​ is not employed at the university any more. The "​Creator"​ is the responsible person in the sense of the Urheberrechtsgesetz,​ taking care that reusing of third party data is legal and in the sense of the DSGVO, that personal data is handled correctly. Even if the "​Creator"​ is not employed at the university any more.
  
- +There exists ​[[https://​doi.org/​10.5281/​zenodo.3368293|decision guide]] if data can be published, sadly only in german.
-If user leaves the university file ownership goes to the next hierarchical user.+
  
 ==== Licensing ==== ==== Licensing ====
Line 376: Line 461:
     * [[https://​www.gnu.org/​licenses/​|GPL,​ GPLv2, GPLv3]]     * [[https://​www.gnu.org/​licenses/​|GPL,​ GPLv2, GPLv3]]
     * [[https://​opensource.org/​licenses/​MIT|MIT]]     * [[https://​opensource.org/​licenses/​MIT|MIT]]
 +    * [[http://​www.linfo.org/​bsdlicense.html|BSD]]
   * Arts, Images, Text, etc.   * Arts, Images, Text, etc.
     * [[https://​creativecommons.org/​choose/​|Creative Commons (Text, Arts, Photos, ...)]]     * [[https://​creativecommons.org/​choose/​|Creative Commons (Text, Arts, Photos, ...)]]
Line 381: Line 467:
     * [[https://​opendatacommons.org/​licenses/​|Open Data Commons]]     * [[https://​opendatacommons.org/​licenses/​|Open Data Commons]]
  
-The applicability of CC-BY licenses for datasets is [[https://​ckan4rdm.wordpress.com/​2019/​06/​05/​creative-commons-lizenzen-sind-fur-forschungsdaten-ungeeignet/​|doubtful]]. ​Other licenses search at [[https://​licenses.opendefinition.org/​|Open Definition Licenses Service]]+ 
 +The applicability of CC-BY licenses ​for Software is not recommended:​ [[https://​creativecommons.org/​about/​program-areas/​software/​|CC-recommendation]] and [[https://​opensource.stackexchange.com/​questions/​1717/​why-is-cc-by-sa-discouraged-for-code|discussion]]. The same applies ​for datasets, their publication under a CC-license other than CC0 is [[https://​ckan4rdm.wordpress.com/​2019/​06/​05/​creative-commons-lizenzen-sind-fur-forschungsdaten-ungeeignet/​|doubtful]]. ​For other dataset ​licenses search at [[https://​licenses.opendefinition.org/​|Open Definition Licenses Service]]
 + 
  
 Proprietary file formats should be avoided, since you don't know if the software to open them still exists in a few years. Try to stick to open standards. Proprietary file formats should be avoided, since you don't know if the software to open them still exists in a few years. Try to stick to open standards.
data_management/irods.1566202634.txt.gz · Last modified: 2019/08/19 10:17 by jsteinka