4
3
2

Hi,

does Maestro have a dependency tree, or a usage report for templates, and components?

I'm trying to do some house keeping and facing challenges.

Some things I'm trying to work out include questions like:

  • which forms use template A
  • which forms use component B

I need the above information to be able to delete some old templates and components. The only way I can find at the moment is to examine every form manually and take notes accordingly. I'm hoping there is a report or similar that will give me a list with this information.


Thanks

Mark

  1. Lin VanOevelen

    Did you ever find a way to do this, Mark?

    We just came across a situation today that requires us to find all forms using a certain component fast.

  2. Mark Murray

    Hi Lin,

    No, unfortunately not. The only solution I have at the moment is basically a manual search, so it's tedious and time consuming.

    I might add this to the 'Suggestion List', for Avoka to include as a feature. Hopefully it will get a few votes and make them aware of the issue.

    Thanks

    Mark

  3. Matthew White

    Maestro does have an export project button which would make this very doable but for me it is greyed out (disabled). Update: Confused projects with Organisation. You can export projects but you don't seem to be able to export the entire organisation (I think that's what it's called).

    Matt

CommentAdd your comment...

3 answers

  1.  
    2
    1
    0

    Hi Mark,

    This groovy script will determine widget usage for current Maestro versions in TM. It may be helpful, please see disclaimer.

    Matt


    import com.avoka.core.groovy.GroovyLogger as logger
    
    /*
    This script assumes that design-info.xml contains a list of all widgets used in a form. I have no idea if this assumption is true.
    Alternatively and probably more reliably you could parse the forms json but that is not included in the far file stored in TM.
    
    DISCLAIMER: This code has minimal testing, no error handling, makes assumptions and was written by a javascript programmer :) Use at your own risk.
    
    Developed in TM 5.1.10
    */
    
    // pretty print an object as json
    def jsp (data) {
        return new groovy.json.JsonBuilder(data).toPrettyString()
    }
    
    // count occurences of a widget in a form
    def countTypes (types) {
        def counts = [:]
        def sortedTypeCounts = [:]
    
        types.each { type ->
            counts[type] = (counts[type] ? counts[type] : 0) + 1
        }
    
        counts.keySet().sort().each { type ->
            sortedTypeCounts[type] = counts[type]
        }
    
        return sortedTypeCounts
    }
    
    // find which forms are used by each widget
    def getFormWidgetUse (widgetUse) {
        def widgets = [:]
        def widgetUseByWidget = [:]
    
        widgetUse.each { e ->
            e.widgetCount.keySet().each { k ->
                widgets[k] = true
            }
        }
    
        def sortedWidgetNames = widgets.keySet().sort()
    
        sortedWidgetNames.each { widget ->
            widgetUseByWidget[widget] = widgetUse
                .findAll { it.widgetCount[widget] > 0 }
                .collect { it.formCode }.sort()
        }
    
        return widgetUseByWidget
    }
    
    // extract design-info.xml out of the far file
    def getDesignInfo (far) {
        def zis = new java.util.zip.ZipInputStream(new ByteArrayInputStream(far))
        def entry = zis.getNextEntry()
    
        while (entry != null) {
            if (entry.getName() == 'META-INF/design-info.xml') {
                def output = new ByteArrayOutputStream()
                output << zis
                zis.closeEntry()
                zis.close()
                return output
            }
            zis.closeEntry()
            entry = zis.getNextEntry()
        }
    
        return
    }
    
    def main () {
        def allClients = (new com.avoka.fc.core.dao.ClientDao()).getAllClients()
        def widgetUse = []
        allClients.each { c ->
            def clientCode = c.getClientCode()
            c.getForms().each { f ->
                def formCode = f.getFormCode()
                def current = f.getCurrentVersion()
                if (current.isMaestroForm()) {
                    def tvd = current.getTemplateVersionData()
                    def far = tvd.getFarFileData()
    
                    if (far != null) {
                        def designXML = getDesignInfo(far)
                        if (designXML) {
                            logger.info("${clientCode} : ${formCode} ${designXML.size()}")
                            def design = new groovy.util.XmlSlurper().parseText(designXML.toString('UTF-8'))
                            def types = design.paths.path.'@type'*.toString()
                            def typeCount = countTypes(types)
                            widgetUse.push([
                                clientCode: clientCode,
                                formCode: formCode,
                                widgetCount: typeCount
                            ])
                        } else {
                            logger.info("${clientCode} : ${formCode} has no design xml")
                        }
                    }
                }
            }
        }
        
        def byWidget = getFormWidgetUse(widgetUse)
    
        def out = [
            byWidget: byWidget,
            widgetUse: widgetUse
        ]
        
        def filename = 'YOUR FILENAME HERE'
    
        (new File(filename)).write(jsp(out))
    
        return 'OK'
    }
    
    return main ()
    
    
    
      CommentAdd your comment...
    1.  
      1
      0
      -1

      Hi Mark,

      That's excellent and very useful to me because I tackled this from the maestro end and can now plug that into the code I wrote for pulling forms out of maestro. If you are interested, the following code will pull down all projects from maestro in the browser and load their json into a single object which can then be queried exactly as you have done.

      To use this, open maestro in chrome and click on your organisation so all your projects are showing. Paste the code into the console, it creates a window object "z" which you can then call functions on to get the data. Simplest use case is:


      z.setup()
      z.loadAll()
      z.conf()

      All the data will be in the object returned from invoking z.conf(). It may break if your projects contain non alpha numeric characters in which case you will need to work out how they translate from a project name to a url.

      If you then want to save that to a single file you can do:

      copy(JSON.stringify(z.conf(), 0, '\t'))
      z = (function (global) {
      	const WARN = true
      	const maestroProjectExportUrl = 'https://maestro.avoka.com.au/maestro/secure/servlet/ProjectV1Servlet/'
      	const cfg = {}
      
      	// helpers
      	const loader = (url) => new Promise((resolve, reject) => {
      		// didn't like using fetch API for some reason so using this instead. Mabye due to version.
      		JSZipUtils.getBinaryContent(url, (err, data) => err ? reject(err) : resolve(data))
      	})
      
      	const zipLoader = async (url) => {
      		if (WARN) console.warn('loading', url);
      		const data = await loader(url)
      		if (WARN) console.warn('completed', url);
      		const o = []
      		const zip = new JSZip();
      		await zip.loadAsync(data)
      		for (let f of zip.file(/./)) {
      			const filename = f.name
      			const content = /\.json/i.test(filename)
      				? JSON.parse(await f.async('text'))
      				: null
      			o.push({ filename, content })
      		}
      		return o
      	}
      
      	const findProject = (input, projects) => 
      		typeof input === 'number'
      			? projects[input]
      			: projects.filter(x => x.name.toLowerCase().trim() == input.toLowerCase().trim())[0]
      
      	const Z = {
      		setup () {
      			// TODO may need additional munging for use in url
      			cfg.org = $('.dashboard-header-info')
      				.text()
      				.toLowerCase()
      				.replace(/^\(/, '')
      				.replace(/\)$/,'')
      
      			cfg.projects = Array.from(
      				$("[data-av-selection='selectedProject']")
      					.map((i,e) => ({
      						name: $(e).find('.dashboard-name-col').text(),
      						version: $(e).find('.dash-right-col').text()
      					}))
      			).map(x => {
      				x.url = maestroProjectExportUrl + cfg.org + '/' + x.name
      					.toLowerCase()
      					.replace(/[/]+/g,'') // TODO maybe other things too
      					.replace(/\s+/g,'-')
      					.replace(/-+/g,'-');
      				return x
      			})
      		},
      		conf () {
      			return cfg
      		},
      		// see comment for 'load'
      		// await z.loadAll().then(z => z.conf())
      		// or await z.loadAll().then(z => z.conf().projects.slice(0,5))
      		async loadAll () {
      			for (let p of cfg.projects.slice(0)) { // take .slice(0) all but allow for easy debugging to slice a range .slice(0,5)
      				p.files = await zipLoader(p.url)
      			}
      			return Z
      		},
      		// you can run this as z.load(3) and you'll get a pending promise to the console or you can use await z.load(3) and get z. This is a bit confusing.
      		// pass the name of the project or its zero based position in the list of projects
      		async load (nameOrPos) {
      			const p = findProject(nameOrPos, cfg.projects)
      			p.files = await zipLoader(p.url)
      			return Z
      		}
      	}
      	return Z
      })(window)
      z.setup()
      
      
      
      1. Mark Murray

        Hi Matt,

        no problem, happy I could help, we both get something out of this one.


        Thanks for the extra code for use in the browser. We only have one 'forms' project at the moment, so it's pretty easy for me to download the whole project.

        I'll keep this code though for future reference.

        I'm a bit pressed fro time at the moment, so this will have to wait for a while before I can give it a lot more attention.

        Thanks again.

        Mark

      CommentAdd your comment...
    2.  
      1
      0
      -1

      Hi Lin, Matthew,

      thanks for the comments and sorry for the delayed reply.

      The good news is I think I have found a way to do this. The idea of downloading the project JSON file was a catalyst.

      I have written a JavaScript file that I can run locally using Node.js, which interrogates the project json file and creates some reports. The code is as follows.

      Note: this has not been audited - I have not verified that it collects all the required information. That will be part of the enhancements as I fine tune it.

      Code:

      #!/usr/bin/env node
      
      // project.js
      /*
      README
      
      This file reads a Maestro project JSON archive to create usage reports.
      
      Usage:
      node project.js
      
      Create a project folder with the following structure:
      Project/
        forms/
        in/
        out/
        src/
      
      Copy the contents of the 'forms' folder from the Maestro archive to the
      'forms' folder of the project.
      Place this file in the src folder.
      
      Run the file using Node.
      
      Reports will be created in the 'out' folder.
      - templates-yyyy-MM-dd.csv
      - styles-yyyy-MM-dd.csv
      - components-yyyy-MM-dd.csv
      
      */
      "use strict";
      
      const fs = require("fs");
      const path = require("path");
      
      let dirForms = "../forms";
      let dirIn = "../in";
      let dirOut = "../out";
      
      // get styles
      function getStyles(form, rows, list) {
          rows.forEach(child => {
              child.forEach(element => {
                  if (element.styles && element.styles.length > 0) {
                      element.styles.forEach(style => {
                          list.push(`${style},${form.label},${element.id},${element.type}`);
                      });
                  }
      
                  if (element.rows && element.rows.length > 0) {
                      getStyles(form, element.rows, list);
                  }
              });
          });
      }
      
      // get custom components
      function getComponents(form, rows, list) {
          rows.forEach(child => {
              child.forEach(element => {
                  if (element.type.startsWith("myKey")) {
                      list.push(`${form.label},${element.type}`);
                  }
      
                  if (element.rows && element.rows.length > 0) {
                      getComponents(form, element.rows, list);
                  }
              });
          });
      }
      
      // clear working directories
      fs.readdirSync(dirIn).forEach(file => {
          fs.unlink(path.join(dirIn, file), err => {
              if (err) throw err;
          });
      });
      
      fs.readdirSync(dirOut).forEach(file => {
          fs.unlink(path.join(dirOut, file), err => {
              if (err) throw err;
          });
      });
      
      // collect 'form.json' definition files
      fs.readdirSync(dirForms).forEach(form => {
          let latest = "";
          fs.readdirSync(path.join(dirForms, form)).forEach(version => {
              if (version > latest) {
                  latest = version;
              }
          })
      
          let content = fs.readFileSync(path.join(dirForms, form, latest, "form.json"));
          let destFile = `${form}.json`
          fs.writeFileSync(path.join(dirIn, destFile), content);
      });
      
      // process definition files
      let templateList = [];
      let styleList = [];
      let componentList = [];
      
      templateList.push("Template Name,Form Name,Brand");
      styleList.push("Style Name,Form Name,Item Name,Item Type");
      componentList.push("Form Name,Component");
      
      fs.readdirSync(dirIn).forEach(definition => {
          let theForm = JSON.parse(fs.readFileSync(path.join(dirIn, definition)));
      
          templateList.push(`${theForm.templateName},${theForm.label},${theForm.selectedBrand}`);
      
          // dialogs
          if (theForm.dialogs.length > 0) {
              theForm.dialogs.forEach(dialog => {
                  getStyles(theForm, theForm.dialog.rows, styleList);
                  getComponents(theForm, theForm.dialog.rows, componentList);
              });
          }
      
          // modals
          if (theForm.modals.length > 0) {
              theForm.modals.forEach(modal => {
                  getStyles(theForm, theForm.modal.rows, styleList);
                  getComponents(theForm, theForm.modal.rows, componentList);
              });
          }
      
          // styles
          if (theForm.styles) {
              Object.entries(theForm.styles).forEach(style => {
                  styleList.push(`${style},${theForm.label},${theForm.id},${theForm.type}`);
              });
          }
      
          // rows
          getStyles(theForm, theForm.rows, styleList);
      
          getComponents(theForm, theForm.rows, componentList);
      });
      
      // output
      let filename = "";
      var today = new Date();
      var yyyy = today.getFullYear();
      var MM = today.getMonth() + 1;
      var dd = today.getDate();
      dd = dd < 10 ? `0${dd}` : dd;
      MM = MM < 10 ? `0${MM}` : MM;
      var filedate = `${yyyy}-${MM}-${dd}`;
      
      filename = `templates-${filedate}.csv`;
      fs.writeFileSync(path.join(dirOut, filename), templateList.join("\n"));
      console.log(`file created: ${filename}`)
      
      filename = `styles-${filedate}.csv`;
      fs.writeFileSync(path.join(dirOut, filename), styleList.join("\n"));
      console.log(`file created: ${filename}`)
      
      filename = `components-${filedate}.csv`;
      fs.writeFileSync(path.join(dirOut, filename), componentList.join("\n"));
      console.log(`file created: ${filename}`)
      
      console.log("... done");


      I hope that is of some help.


      I still hope that Avoka will come to the party and build this feature in to the product. Now that I have this code I can use it again, but it took a while to put together, which is time I'd rather spend doing other things.


      Regards

      Mark

        CommentAdd your comment...