Expect

Last updated
Expect
Original author(s) Don Libes
Developer(s) Nils Carlson
Stable release
5.45.4 / 4 February 2018;5 years ago (2018-02-04)
Written in C
Operating system POSIX, Windows
License Public domain [1]
Website core.tcl-lang.org/expect/

Expect is an extension to the Tcl scripting language written by Don Libes. [2] The program automates interactions with programs that expose a text terminal interface. Expect, originally written in 1990 for the Unix platform, has since become available for Microsoft Windows and other systems.

Contents

Basics

Expect is used to automate control of interactive applications such as Telnet, FTP, passwd, fsck, rlogin, tip, SSH, and others. [3] Expect uses pseudo terminals (Unix) or emulates a console (Windows), starts the target program, and then communicates with it, just as a human would, via the terminal or console interface. [4] Tk, another Tcl extension, can be used to provide a GUI. [5]

Usage

Expect serves as a "glue" to link existing utilities together. The general idea is to figure out how to make Expect use the system's existing tools rather than figure out how to solve a problem inside of Expect.

A key usage of Expect involves commercial software products. Many of these products provide some type of command-line interface, but these usually lack the power needed to write scripts. They were built to service the users administering the product, but the company often does not spend the resources to fully implement a robust scripting language. An Expect script can spawn a shell, look up environmental variables, perform some Unix commands to retrieve more information, and then enter into the product's command-line interface armed with the necessary information to achieve the user's goal. After retrieving information by interacting with the product via its command-line interface, the script can make intelligent decisions about what action to take, if any.

Every time an Expect operation is completed, the results are stored in a local variable called $expect_out. This allows the script to harvest information to feedback to the user, and it also allows conditional behavior of what to send next based on the circumstances.

A common use of Expect is to set up a testing suite for programs, utilities or embedded systems. DejaGnu is a testing suite written using Expect for use in testing. It has been used for testing GCC and remote targets such as embedded development.

Expect script can be automated using a tool called 'autoexpect'. This tool observes your actions and generates an Expect script using heuristics. Though generated code may be large and somewhat cryptic, one can always tweak the generated script to get the exact code.

# Assume $remote_server, $my_user_id, $my_password, and # $my_command were read earlier in the script.# Open a Telnet session to a remote server, and wait # for a username prompt.spawntelnet$remote_serverexpect"username:"# Send the username, and then wait for a password prompt.send"$my_user_id\r"expect"password:"# Send the password, and then wait for a shell prompt.send"$my_password\r"expect"%"# Send the prebuilt command, and then wait # for another shell prompt.send"$my_command\r"expect"%"# Capture the results of the command into a variable. This # can be displayed, or written to disk.setresults$expect_out(buffer)# Exit the Telnet session, and wait for a special # end-of-file character.send"exit\r"expecteof 

Another example is a script that automates FTP:

# Set timeout parameter to a proper value.# For example, the file size is indeed big and the network # speed is really one problem, you'd better set this # parameter a value.settimeout-1# Open an FTP session to a remote server, and # wait for a username prompt.spawnftp$remote_serverexpect"username:"# Send the username, and then wait for a password prompt.send"$my_user_id\r"expect"password:"# Send the password, and then wait for an 'ftp' prompt.send"$my_password\r"expect"ftp>"# Switch to binary mode, and then wait for an 'ftp' prompt.send"bin\r"expect"ftp>"# Turn off prompting.send"prompt\r"expect"ftp>"# Get all the filessend"mget *\r"expect"ftp>"# Exit the FTP session, and wait for a special # end-of-file character.send"bye\r"expecteof 

Below is an example that automates SFTP (with a password):

#!/usr/bin/env expect -f# Procedure to attempt connecting; result 0 if OK, 1 otherwiseprocconnect{passw}{expect{"Password:"{send"$passw\r"expect{"sftp*"{return0}}}}# Timed outreturn1}# Read the input parameterssetuser[lindex$argv0]setpassw[lindex$argv1]sethost[lindex$argv2]setlocation[lindex$argv3]setfile1[lindex$argv4]setfile2[lindex$argv5]#puts "Argument data:\n";#puts "user: $user";#puts "passw: $passw";#puts "host: $host";#puts "location: $location";#puts "file1: $file1";#puts "file2: $file2";# Check if all were providedif{$user==""||$passw==""||$host==""||$location==""||$file1==""||$file2==""}{puts"Usage: <user> <passw> <host> <location> <file1 to send> <file2 to send>\n"exit1}# Sftp to specified host and send the filesspawnsftp$user@$hostsetrez[connect$passw]if{$rez==0}{send"cd $location\r"settimeout-1send"put $file2\r"send"put $file1\r"send"ls -l\r"send"quit\r"expecteof exit0}puts"\nError connecting to server: $host, user: $user and password: $passw!\n"exit1

Using passwords as command-line arguments, like in this example, is a huge security hole, as any other user on the machine can read this password by running "ps". You can, however, add code that will prompt you for your password rather than giving your password as an argument. This should be more secure. See the example below.

stty-echo send_user--"Enter Password: "expect_user-re"(.*)\n"send_user"\n"sttyecho setPASS$expect_out(1,string)

Another example of automated SSH login to a user machine:

# Timeout is a predefined variable in Expect which by # default is set to 10 seconds.# spawn_id is another predefined variable in Expect.# It is a good practice to close spawn_id handle # created by spawn command.settimeout60spawnssh$user@machine while{1}{expect{eof{break}"The authenticity of host"{send"yes\r"}"password:"{send"$password\r"}"*\]"{send"exit\r"}}}waitclose$spawn_id

Alternatives

Various projects implement Expect-like functionality in other languages, such as C#, Java, Scala, Groovy, Perl, Python, Ruby, Shell and Go. These are generally not exact clones of the original Expect, but the concepts tend to be very similar.

C#

Erlang

Go

Groovy

Java

Perl

Python

Ruby

Rust

Scala

Shell

Related Research Articles

<span class="mw-page-title-main">AWK</span> Programming language

AWK is a domain-specific language designed for text processing and typically used as a data extraction and reporting tool. Like sed and grep, it is a filter, and is a standard feature of most Unix-like operating systems.

The File Transfer Protocol (FTP) is a standard communication protocol used for the transfer of computer files from a server to a client on a computer network. FTP is built on a client–server model architecture using separate control and data connections between the client and the server. FTP users may authenticate themselves with a plain-text sign-in protocol, normally in the form of a username and password, but can connect anonymously if the server is configured to allow it. For secure transmission that protects the username and password, and encrypts the content, FTP is often secured with SSL/TLS (FTPS) or replaced with SSH File Transfer Protocol (SFTP).

AppleScript is a scripting language created by Apple Inc. that facilitates automated control over scriptable Mac applications. First introduced in System 7, it is currently included in all versions of macOS as part of a package of system automation tools. The term "AppleScript" may refer to the language itself, to an individual script written in the language, or, informally, to the macOS Open Scripting Architecture that underlies the language.

curl is a computer software project providing a library (libcurl) and command-line tool (curl) for transferring data using various network protocols. The name stands for "Client for URL".

chroot is an operation on Unix and Unix-like operating systems that changes the apparent root directory for the current running process and its children. A program that is run in such a modified environment cannot name files outside the designated directory tree. The term "chroot" may refer to the chroot(2) system call or the chroot(8) wrapper program. The modified environment is called a chroot jail.

Secure copy protocol (SCP) is a means of securely transferring computer files between a local host and a remote host or between two remote hosts. It is based on the Secure Shell (SSH) protocol. "SCP" commonly refers to both the Secure Copy Protocol and the program itself.

The Berkeley r-commands are a suite of computer programs designed to enable users of one Unix system to log in or issue commands to another Unix computer via TCP/IP computer network. The r-commands were developed in 1982 by the Computer Systems Research Group at the University of California, Berkeley, based on an early implementation of TCP/IP.

In the context of an HTTP transaction, basic access authentication is a method for an HTTP user agent to provide a user name and password when making a request. In basic HTTP authentication, a request contains a header field in the form of Authorization: Basic <credentials>, where credentials is the Base64 encoding of ID and password joined by a single colon :.

Filesystem in Userspace (FUSE) is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running file system code in user space while the FUSE module provides only a bridge to the actual kernel interfaces.

<span class="mw-page-title-main">Comparison of command shells</span>

A command shell is a command-line interface to interact with and manipulate a computer's operating system.

test is a command-line utility found in Unix, Plan 9, and Unix-like operating systems that evaluates conditional expressions. test was turned into a shell builtin command in 1981 with UNIX System III and at the same time made available under the alternate name [.

In computing, exec is a functionality of an operating system that runs an executable file in the context of an already existing process, replacing the previous executable. This act is also referred to as an overlay. It is especially important in Unix-like systems, although it exists elsewhere. As no new process is created, the process identifier (PID) does not change, but the machine code, data, heap, and stack of the process are replaced by those of the new program.

<span class="mw-page-title-main">Areca Backup</span> File backup system

Areca Backup is an Open Source personal file backup software developed in Java. It is released under the GNU General Public License (GPL) 2.

tip is a Unix utility for establishing a terminal connection to a remote system via a modem. It is commonly associated with BSD Unix, as well as other UNIX operating systems such as Sun's Solaris. It was originally included with 4.2BSD. The name may refer to ARPANET's Terminal Interface Processor (TIP), a variant of the IMP, used to connect serial terminals directly with ARPANET.

CrushFTP is a proprietary multi-protocol, multi-platform file transfer server originally developed in 1999. CrushFTP is shareware with a tiered pricing model. It is targeted at home users on up to enterprise users.

.htpasswd is a flat-file used to store usernames and password for basic authentication on an Apache HTTP Server. The name of the file is given in the .htaccess configuration, and can be anything although ".htpasswd" is the canonical name. The file name starts with a dot, because most Unix-like operating systems consider any file that begins with dot to be hidden. The "htpasswd" command is used to create and manage ".htpasswd" file. With it, you can securely add, remove, and modify user entries, all while ensuring the passwords are securely encoded.

GVfs is GNOME's userspace virtual filesystem designed to work with the I/O abstraction of GIO, a library available in GLib since version 2.15.1. It installs several modules that are automatically used by applications using the APIs of libgio. There is also FUSE support that allows applications not using GIO to access the GVfs filesystems.

Gumblar is a malicious JavaScript trojan horse file that redirects a user's Google searches, and then installs rogue security software. Also known as Troj/JSRedir-R this botnet first appeared in 2009.

<span class="mw-page-title-main">Command-line interface</span> Computer interface that uses text

A command-line interface (CLI) is a means of interacting with a computer program by inputting lines of text called command-lines. Command-line interfaces emerged in the mid-1960s, on computer terminals, as a user-friendly alternative to punched cards.

References

  1. "Expect FAQ: Our company policy requires a license to use Expect. Where can we get a license?". 2006-10-11.
  2. Nemeth, Evi; Snyder, Garth; Seebass, Scott; Hein, Trent (2000-08-29). UNIX System Administration Handbook. Pearson Education. ISBN   978-0-13-700291-7.
  3. Mckay, David (May 24, 2021) [May 24, 2021]. "Automate Inputs to Linux Scripts With the expect Command". category/Linux. Retrieved Nov 28, 2023.{{cite web}}: CS1 maint: date and year (link)
  4. Carling, M.; Degler, Stephen; Dennis, James (2000). Linux System Administration. Sams Publishing. ISBN   978-1-56205-934-7.
  5. "Tcl/Tk in the Development of User-Extensible Graphical User Interfaces". www.usenix.org. Retrieved 2022-08-11.

Further reading