Dataset: Group of structured data retrievable in a link or single instruction as a whole to a single entity, with updating frequency larger than a once a minute.
Structured: The data has to be structured and therefore the relation between the elements has to be clearly and unequivocally determined.
Data: Meaning that the contents are digitally codified.
Link or single instruction: The term dataset is clearly oriented to an online world, where the term link really applies, however the definition would also work for other environments where single instruction would apply.
This link or single instruction ranges from a rather simple URL to a more complex SPARQL query.
To a single entity. There are some querying languages (i.e. SPARQL) that are capable to retrieve information from multiple sources, once you have to identify different sources it cannot be considered a single dataset.
As a whole: It is required that you actually can access to the full content of the dataset. If it cannot be retrieved in a single link cannot be defined as a single dataset.
Updating frequency: The possible change rate. Potentially the dataset could change completely between iterations. It does not mean that it is compulsory to change on every update but that the dataset is potentially fully updated with that frequency.