Adding Text to Canvas via script

I have added an empty canvas to the scene called myCanvas and attached TestScript to it.

Tell me please, how can one add a GUI-text object using a C# script?

using UnityEngine;
using UnityEngine.UI;
using System.Collections;

public class TestScript : MonoBehaviour {
  void Start () {
    // gameObject represent myCanvas

    // add myText...
  }
}

The Canvas is a GameObject, so you can manipulate it just like you would for a 3D object.

The best way to add text would be to add an empty GameObject to Canvas then add the text component to that new GameObject.

void Start() {
   GameObject newGO = new GameObject("myTextGO");
   ngo.transform.SetParent(this.transform);

   Text myText = ngo.AddComponent<Text>();
   myText.text = "Ta-dah!";
}

Unity 5 example

public static Text AddTextToCanvas(string textString, GameObject canvasGameObject)
{
    Text text = canvasGameObject.AddComponent<Text>();
    text.text = textString;

    Font ArialFont = (Font)Resources.GetBuiltinResource(typeof(Font), "Arial.ttf");
    text.font = ArialFont;
    text.material = ArialFont.material;

    return text;
}

What version of Unity are you using?
If your using Unity5 then you should use UnityEngine.UI.Text instead of the GUIText Class (before Unity 4.6).

Text myText = new Text();

Adding any Object as a child is simple just attach this script to the GameObject that you want to make Parent. You will need to assign Font for the text you want to show, as it is not assigned internally. Just attach this to some GameObject in Canvas. If you want to attach this to a GameObject outside Canvas change this.transform to your particular GameObject that is public (like myGameObject.transformand assign it in Inspector,

public void AddChild()
{
     GameObject gameObject = new GameObject("Child");
     gameObject.transform.SetParent(this.transform);

     gameObject .AddComponent<Text>().text = "Hello This is Child";
     gameObject .GetComponent<Text>().font = Resources.GetBuiltinResource(typeof(Font), "Arial.ttf") as Font;
}