Compressing directory with to Zip file with Golang

Golang version 1.23.6 supporting tar and zip compressing. In this article I will show you how to compress directory with Golang

Creating a output file

First we need to create output file.

filename := "test.zip"
output_file, err := os.Create(filename)
if err != nil {
	panic(err)
}

if you want to compress with tar file, you can replace extension to the .tar

Creating a writer

w := zip.NewWriter(output_file)
defer w.Close()

creating new file under the zip stream

path := "helloworld.txt"
zipFile, err := w.Create(path)
if err != nil {
	panic(err)
}

Writing to the zip stream

myBytes := []byte("hello world !")
zipFile.Write(myBytes)

let's see output

> unzip -l test.zip

  Length      Date    Time    Name
---------  ---------- -----   ----
       13  00-00-1980 00:00   helloworld.txt
---------                     -------
       13                     1 file

>  zipinfo test.zip

Archive:  test.zip
Zip file size: 161 bytes, number of entries: 1
-rw----     2.0 fat       13 bl defN 80-000-00 00:00 helloworld.txt
1 file, 13 bytes uncompressed, 19 bytes compressed:  -46.2%

> unzip -p test.zip helloworld.txt 

hello world !%

Writing header metadata

CreateHeader method provides creating new zip file with metadata in stream

zipFile, err := w.CreateHeader(&zip.FileHeader{
	Name:     path,
	Comment:  "Test Comment",
	Modified: time.Now()},
)

let's check out again the zip file :

> zipinfo test.zip

Archive:  test.zip
Zip file size: 185 bytes, number of entries: 1
-rw----     2.0 fat       13 bX stor 25-Feb-07 14:35 helloworld.txt
1 file, 13 bytes uncompressed, 13 bytes compressed:  0.0%

we can see that file modification date is updated

⇥ Tutorials ⇥ Golang

What is robots.txt, why we need it ?

During the first time search engine bots visiting website, they check domain root directory. Robots.txt file contains information which sub directories should be indexed or not

Robots.txt example

https://www.google.com/robots.txt

Robots.txt structure

Robots.txt has basic structure

User-agent: useragent1
Disallow: ...
Allow: ...
Sitemap: ...

line starts with User-agent informs search engine regarding to valid operations on below

for example

User-agent: *
Disallow: /search
Allow: /search/about

this block indicates that /search path is disallowed for all clients and /search/about path is allowed

Allowing all pages for all user agents

User-agent: *
Allow: *

Disallowing all pages for all user agents

User-agent: *
Disallow: *

Sitemap tag

sitemap tag shows where sitemap file located

Sitemap: https://www.google.com/sitemap.xml

there should be only one Sitemap tag in robots.txt file.

we recommend using sitemap index file in the multiple sitemap case

⇥ Tutorials ⇥ Golang

Generating xml sitemap with Golang

sitemaps are xml based files shared for search engines. Main purpose of sitemap is making websites easy indexable

golang library encoding/xml contains functions for marshalling & unmarshalling xml data format

Sitemap xml structure

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
      <loc>http://33m.org/</loc>
      <lastmod>2024-12-01</lastmod>
      <changefreq>monthly</changefreq>
      <priority>0.9</priority>
   </url>
</urlset>

Urlset tag is root tag of the sitemap file that we can add more than one url's under.

Url tag contains :

loc tag indicates page url (required)

lastmod tag indicates last modification date

changefreq indicates page update frequency

priority tag indicates page priority

only loc tag is required, other tags are optional

Sitemap Index file

if we have multiple sitemap, we can add all sitemaps to the one sitemap index file

sitemap index file structure

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <sitemap>
      <loc>http://33m.org/sitemap.xml</loc>
      <lastmod>2024-12-12T18:23:17+00:00</lastmod>
   </sitemap>
   <sitemap>
      <loc>http://33m.org/sitemap2.xml</loc>
      <lastmod>2024-12-22</lastmod>
   </sitemap>
</sitemapindex>

tags under the sitemap tag have same function with url tag.

only we need to add sitemap url

Golang encoding/xml library

encoding/xml library supports Marshalling & Unmarshalling xml

lets create a example sitemap with Golang

Declaration of object structures for sitemap

type UrlSet struct {
	XMLName           xml.Name `xml:"urlset"`
	Url               []Url    `xml:"url"`
	Xsi               string   `xml:"xmlns:xsi,attr"`
	XsiSchemaLocation string   `xml:"xsi:schemaLocation,attr"`
	Xmlns             string   `xml:"xmlns,attr"`
}

type Url struct {
	Loc     string  `xml:"loc"`
	Lastmod *string `xml:"lastmod"`
}

Creating Instance of Urlset Object

locs := []Url{}
locs = append(locs, Url{Loc: "http://33m.org/golang", Lastmod: "2024-12-12"})
locs = append(locs, Url{Loc: "http://33m.org/java", Lastmod: "2024-11-12"})
locs = append(locs, Url{Loc: "http://33m.org/linux", Lastmod: "2024-11-12"})

urlSet := UrlSet{
		Url:               locs,
		Xmlns:             "http://www.sitemaps.org/schemas/sitemap/0.9",
		XsiSchemaLocation: "http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd",
		Xsi:               "http://www.w3.org/2001/XMLSchema-instance",
	}

Using different namespaces are possible, we use default sitemap schema (http://www.sitemaps.org/schemas/sitemap/0.9)

Marshalling

xmlBytes, err := xml.Marshal(urlSet)
if err != nil {
	panic(err)
}

fmt.Println(string(xmlBytes))

Result :

<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
    xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>http://33m.org/golang</loc>
        <lastmod>2024-12-12</lastmod>
    </url>
    <url>
        <loc>http://33m.org/java</loc>
        <lastmod>2024-12-12</lastmod>
    </url>
    <url>
        <loc>http://33m.org/linux</loc>
        <lastmod>2024-12-12</lastmod>
    </url>
</urlset>

Declaration of object structures for sitemap index

type SitemapIndex struct {
	XMLName           xml.Name  `xml:"sitemapindex"`
	Sitemap           []Sitemap `xml:"sitemap"`
	Xsi               string    `xml:"xmlns:xsi,attr"`
	XsiSchemaLocation string    `xml:"xsi:schemaLocation,attr"`
	Xmlns             string    `xml:"xmlns,attr"`
}

type Sitemap struct {
	Loc     string  `xml:"loc"`
	Lastmod *string `xml:"lastmod"`
}

Creating instance of SitemapIndex Object

sitemaps := []Sitemap{}
sitemaps = append(sitemaps, Sitemap{Loc: "http://33m.org/sitemap1.xml"})
sitemaps = append(sitemaps, Sitemap{Loc: "http://33m.org/sitemap2.xml"})


sitemapIndex := SitemapIndex{
		Sitemap:           sitemaps,
		Xmlns:             "http://www.sitemaps.org/schemas/sitemap/0.9",
		XsiSchemaLocation: "http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd",
		Xsi:               "http://www.w3.org/2001/XMLSchema-instance",
	}

Marshalling

xmlBytes, err := xml.Marshal(sitemapIndex)
if err != nil {
	panic(err)
}

fmt.Println(string(xmlBytes))

Result :

<sitemapindex xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
    xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <sitemap>
        <loc>http://33m.org/sitemap1.xml</loc>
    </sitemap>
    <sitemap>
        <loc>http://33m.org/sitemap2.xml</loc>
    </sitemap>
</sitemapindex>

⇥ Tutorials ⇥ Golang

Reverse integer algorithm, how programming language reverse strings ?

Reversing strings

strings are character arrays, first we need to understand it

Js reverse command is easy

var str="asdf";
console.log(str.reverse());

⇥ Tutorials ⇥ Code Snippets ⇥ Javascript ⇥ Algorithms ⇥ Leetcode