When you start working with Golang, you might see a struct definition like the following.
type employee struct {
Name string `json:"name"`
Age int `json:"age,omitempty"`
}It absolutely looks like JSON-related things but how it actually works?
If you need to parse or convert JSON because your application communicates with REST API or something similar, this post is for you.
- How to convert from struct to json and vice versa
- Define the name usesd in the json
- Is a key-value assigned to struct when the json name is different?
- How to assign null to json
- How to omit the key when the value is not set
- How to ignore key
- How to create nested json structure
- How to Unmarshal without struct definition
- Related article
How to convert from struct to json and vice versa
Golang offers a simple way to convert. Let’s define the following simple struct.
type product struct {
Name string
Price int
}struct to json
json.Marshal(json) can be the first option.
item := product{
Name: "Apple",
Price: 55,
}
jsonBytes, err := json.Marshal(item)
if err != nil {
fmt.Println(err)
}
fmt.Println(string(jsonBytes))
// {"Name":"Apple","Price":55}The json.Marshal returns byte array. So it’s necessary to convert it to string.
json to struct with indent
Use json.MarshalIndent() instead if you want to make it more readable.
prefix := ""
indent := "\t"
jsonBytes2, err := json.MarshalIndent(item, prefix, indent)
fmt.Println(string(jsonBytes2))
// {
// "Name": "Apple",
// "Price": 55
// }This is useful when you want to write the content of the struct to log.
json to struct
Use json.Unmarshal(jsonBytes, *dest) to convert JSON to struct.
var fromJson []product
myJson := []byte(`[
{"Name": "Apple", "prICe": 11, "id": 3},
{"Name": "Melon", "PricE": 22, "id": 4}
]`)
err = json.Unmarshal(myJson, &fromJson)
if err != nil {
fmt.Println(err)
}
fmt.Printf("%+v\n", fromJson)
// [{Name:Apple Price:11} {Name:Melon Price:22}]The second parameter needs to be a pointer.
As you can see, the undefined key value is not assigned to the struct. However, it is not case-sensitive. Even though one of the key letters’ cases is different, the value is correctly assigned.
Define the name usesd in the json
If you want to define the name used in JSON, you can define it on the same line as the variable in a struct. The format is the following.
json:"key_name"
Let’s see this example.
type employee struct {
Name string "json:name"
Age int `json:"age"`
Gender string `json:"gender"`
Job string `json:"role"`
WithoutSchema string
}
employee := employee{
Name: "Yuto",
Age: 35,
Gender: "Male",
Job: "Software Developer",
}
prefix := ""
indent := "\t"
jsonBytes2, _ := json.MarshalIndent(employee, prefix, indent)
fmt.Println(string(jsonBytes2))
// {
// "Name": "Yuto",
// "age": 35,
// "gender": "Male",
// "role": "Software Developer",
// "WithoutSchema": ""
// }The Name variable doesn’t have double quotes for the key itself. Therefore, the variable name is used in the JSON as it is.
Age and Gender have the expected format, so the first letter is lowercase. The name of Job in JSON is totally different but it’s also written as expected.
As you saw in the previous section, the variable name is used since it has no schema definition for WithoutSchema.
Is a key-value assigned to struct when the json name is different?
What happens if the two struct has the same variable name but a different JSON name? Let’s define the following struct. Name has all the lower cases but the same letter. Gender has totally different naming.
type person struct {
Name string `json:"name"`
Age int `json:"age"`
Gender string `json:"fooo"`
}Then, let’s try to convert a JSON to struct.
employee := employee{
Name: "Yuto",
Age: 35,
Gender: "Male",
Job: "Software Developer",
}
jsonBytes, _ := json.Marshal(employee)
var yuto person
err := json.Unmarshal(jsonBytes, &yuto)
if err != nil {
fmt.Println(err)
}
fmt.Printf("%+v\n", yuto)
// {Name:Yuto Age:35 Gender:}
fmt.Printf("Name: %s, Age: %d, Gender: %s\n", yuto.Name, yuto.Age, yuto.Gender)
// Name: Yuto, Age: 35, Gender:The destination name is fooo while the JSON object has gender hence the key value is not assigned to the Gender property.
How to assign null to json
If the variable is defined as a non-pointer type, the default value is assigned when no data is specified.
If you want to assign null to the key, you need to define the variable as a pointer.
type jsonTestStruct struct {
// add asterisk to the data type
StringPointer0 *string `json:"stringPointer0"`
StringPointer1 *string `json:"stringPointer1"`
}
str := ""
obj := jsonTestStruct{
StringPointer0: &str,
StringPointer1: nil,
}
jsonBytes, _ := json.MarshalIndent(obj, prefix, indent)
fmt.Println(string(jsonBytes))
// {
// "stringPointer0": "",
// "stringPointer1": null,
// } You don’t have to explicitly set nil.
How to omit the key when the value is not set
There are some cases where the key should not appear in the JSON if the value is not specified. In this case, you can add omitempty keyword in the schema.
type jsonTestStruct struct {
StringPointer0 *string `json:"stringPointer0"`
StringPointer1 *string `json:"stringPointer1"`
StringPointer2 *string `json:"stringPointer2,omitempty"`
StringValue string `json:"stringValue,omitempty"`
}It must be in the double quotes. Don’t add a space after the comma.
str := ""
obj := jsonTestStruct{
StringPointer2: &str,
StringValue: "",
}
jsonBytes, _ := json.MarshalIndent(obj, prefix, indent)
fmt.Println(string(jsonBytes))
// {
// "stringPointer0": null,
// "stringPointer1": null,
// "stringPointer2": "",
// }stringPointer2 has an empty string there because it is a pointer type. On the other hand, StringValue is omitted. According to the official documentation, if the value is the default value, the key is omitted.
The “omitempty” option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.
https://pkg.go.dev/encoding/json#Unmarshal
If you want to differentiate between a default value and null, you need to define the variable as a pointer type.
Let’s test a little bit more for those types described above.
type jsonTestDefault struct {
IntValue int `json:"intValue,omitempty"`
IntArray []int `json:"intArray,omitempty"`
BoolValue bool `json:"boolValue,omitempty"`
MapValue map[int]string `json:"mapValue,omitempty"`
}
obj := jsonTestDefault{
IntValue: 0,
IntArray: []int{},
BoolValue: false,
MapValue: map[int]string{},
}
jsonBytes, _ := json.MarshalIndent(obj, prefix, indent)
fmt.Println(string(jsonBytes))
// {}The result is as expected.
obj := jsonTestDefault{
IntValue: -1,
IntArray: []int{0},
BoolValue: true,
MapValue: map[int]string{0: ""},
}
jsonBytes, _ := json.MarshalIndent(obj, prefix, indent)
fmt.Println(string(jsonBytes))
// {
// "intValue": -1,
// "intArray": [
// 0
// ],
// "boolValue": true,
// "mapValue": {
// "0": ""
// }
// }When the value is set, all of them are set correctly.
How to ignore key
If there are some credentials in the struct, you might not want to add them to the JSON. In this case, define the name as dash “-“.
type jsonTestOther struct {
AsIs int `json:""`
Ignored string `json:"-"`
Dash string `json:"-,"`
}
obj := jsonTestOther{
AsIs: 11,
Ignored: "this is ignored",
Dash: "this is not ignored",
}
jsonBytes, _ := json.MarshalIndent(obj, prefix, indent)
fmt.Println(string(jsonBytes))
// {
// "AsIs": 11,
// "-": "this is not ignored"
// }Note that the key is not ignored if you add a comma followed by a dash.
How to create nested json structure
You might already have the idea. You need to just define a struct where one of the keys has another struct.
type department struct {
Name string `json:"name"`
Members []employee `json:"members,omitempty`
}
type employee struct {
Name string "json:name"
Age int `json:"age"`
Gender string `json:"gender"`
Job string `json:"role"`
WithoutSchema string
}Then, the rest is the same.
employees := []employee{
{
Name: "Yuto",
Age: 35,
Gender: "Male",
Job: "Software Developer",
},
{
Name: "Daniel",
Age: 40,
Gender: "Male",
Job: "Software Tester",
},
}
department := department{
Name: "Technical Feeder",
Members: employees,
}
jsonBytes, _ := json.MarshalIndent(department, prefix, indent)
fmt.Println(string(jsonBytes))
// {
// "name": "Technical Feeder",
// "Members": [
// {
// "Name": "Yuto",
// "age": 35,
// "gender": "Male",
// "role": "Software Developer",
// "WithoutSchema": ""
// },
// {
// "Name": "Daniel",
// "age": 40,
// "gender": "Male",
// "role": "Software Tester",
// "WithoutSchema": ""
// }
// ]
// }How to Unmarshal without struct definition
There is a case where it’s impossible to define a struct in advance. How can we unmarshal the JSON data in this case? We can use map.
Let’s prepare a map object and the JSON data.
var objMap map[string]any
if err := json.Unmarshal([]byte(`{
"name": "Yuto",
"prop1": 12,
"prop2": 2.2,
"prop3": false,
"prop4": null,
"prop5": [1,2,3,"str"],
"prop6": {
"prop6-nested": 15
}
}`), &objMap); err != nil {
fmt.Println(err)
return
}The JSON data contains all the possible data types. The key in the JSON data is assigned to the map key.
How to read literal values
The value for string, number, bool, and null can be easily read by specifying the key name in the map.
func showData(data any) {
dataType := reflect.TypeOf(data).Kind().String()
fmt.Printf("type: %s, value: %+v\n", dataType, data)
}
showData(objMap) // type: map, value: map[name:Yuto prop1:12 prop2:2.2 prop3:false prop4:<nil> prop5:[1 2 3 str] prop6:map[prop5-nested:15]]
showData(objMap["name"]) // type: string, value: Yuto
showData(objMap["prop1"]) // type: float64, value: 12
fmt.Println(objMap["prop1"] == 12) // false
fmt.Println(objMap["prop1"] == float64(12)) // true
showData(objMap["prop2"]) // type: float64, value: 2.2
showData(objMap["prop3"]) // type: bool, value: false
fmt.Println(objMap["prop4"]) // <nil>Note that all numbers are converted to float64. Both 12 and 2.2 are float64. Therefore, if it’s necessary to compare the value with int value, one of the values must be converted to the proper data type.
How to read an Array data
If the data type is Array, we need additional steps. It can’t be used in for with range keyword because the data type is any.
// cannot range over array (variable of type any)
for index, data := range array {}It’s possible to read the whole data but not possible to read data with index array.
array := objMap["prop5"]
showData(array) // type: slice, value: [1 2 3,str]
// invalid operation: cannot index array (variable of type any)
showData(array[0])If the specific data needs to be read, it has to be accessed in the following way.
valueOfArray := reflect.ValueOf(array)
showData(valueOfArray.Index(0).Interface()) // type: float64, value: 1
showData(valueOfArray.Index(0).Elem().Float()) // type: float64, value: 1
showData(valueOfArray.Index(1).Interface()) // type: float64, value: 2
showData(valueOfArray.Index(1).Elem().Float()) // type: float64, value: 2
showData(valueOfArray.Index(2).Interface()) // type: float64, value: 3
showData(valueOfArray.Index(2).Elem().Float()) // type: float64, value: 3
showData(valueOfArray.Index(3).Interface()) // type: string, value: str
showData(valueOfArray.Index(3).Elem().String()) // type: string, value: strIf you just need the value without the data type, use Interface(). If the value needs to be passed to a function that defines a concrete data type, the data should be read by Elem().Float() for example. It depends on the data which method should be used. If it is a string, use Elem().String() instead.
Check if the data is Slice and the length of the slice if Index() needs to be used. Otherwise, it panics.
fmt.Println(valueOfArray.Kind().String()) // slice
fmt.Println(valueOfArray.Kind() == reflect.Slice) // true
showData(valueOfArray.Len()) // type: int, value: 4If the array needs to be iterated over, use a switch case to check the data type. range can be used in the case statement.
func handleArrayAndMap(array any) {
fmt.Println("--- handleArray start ---")
switch v := array.(type) {
case []any:
for _, data := range v {
showData(data)
}
case map[string]any:
for _, data := range v {
showData(data)
}
default:
fmt.Println("not an array/map")
}
fmt.Println("--- handleArray end ---")
}
handleArrayAndMap(array)
// type: float64, value: 1
// type: float64, value: 2
// type: float64, value: 3
// type: string, value: strThe function can be used for map too.
How to read nested object data
If a JSON value is an object, it is mapped to map. prop6 is an object.
"prop6": {
"prop6-nested": 15
}The whole value can be easily read. If it’s necessary to iterate the items, we can write it in the same way as array. See the above about the detail for handleArrayAndMap.
nestedObj := objMap["prop6"]
showData(nestedObj) // type: map, value: map[prop6-nested:15]
handleArrayAndMap(nestedObj) // key: prop6-nested, value: 15If reflect.ValueOf() is used for the map value, it is converted to a struct. The value can be accessed via MapIndex() with reflect.ValueOf("key-name").
valueOfNestedObj := reflect.ValueOf(nestedObj)
showData(valueOfNestedObj) // type: struct, value: map[prop6-nested:15]
showData(valueOfNestedObj.MapIndex(reflect.ValueOf("prop6-nested"))) // type: float64, value: 15Another way to read the data is to cast it to map.
convertedMap, ok := nestedObj.(map[string]any)
if !ok {
fmt.Println("conversion error for map")
} else {
showData(convertedMap["prop6-nested"]) // type: float64, value: 15
}The data can be read with the key name if the cast succeeds. Of course, it’s possible to loop the map in this way.
Comments